CoChat
Sign Up
Cited by 33Expert finding and Q&A systemsInnovative Human-Technology InteractionUsability and User Interface Design

Critique Me: Exploring How Creators Publicly Request Feedback in an Online Critique Community

Open access
Cheng, Ruijia, Zeng, Ziwen, Liu, Maysnow, Dow, Steven·2020·Proceedings of the ACM on Human-Computer Interaction
DOI 10.1145/3415232
Open this paper in CoChat — free accountOpen PDF

Abstract

Creative workers frequently turn to online critique communities for feedback on their work. While past research has focused primarily on how to yield better feedback from providers, less is known about the strategies feedback seekers use to engage providers and request feedback. We present two studies to explore the feedback exchange dynamics between feedback requesters and providers in the subreddit community, r/design\_critiques. In Study 1, we interviewed 12 community members and found that while creators have strategies to request feedback, they expressed uncertainty about whether and how to include details about the design context, personal background, and specific feedback needs. In Study 2, through a mixed-method analysis, we identified how specific request strategies impact the quantity and quality of community feedback, and found several key, but undervalued strategies: signaling as a novice, critiquing one's own design, and providing design variants. These strategies led to better community response, but were rarely used. We offer design implications around how to leverage these insights to improve online feedback exchange

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3415232

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Our sample of 26 exceeds the power analysis bound and aligns with prior HCI work on AI-supported evaluation and critique.

    “Our sample of 26 exceeds this bound and aligns with prior HCI work on AI-supported evaluation and critique [11, 17, 88].”

    From Criticmate: Stagewise Human–AI Co-Critique in Single-Screen UI Evaluation (2026) · cites this paper as [11]

    Section 4.1 (page unavailable) describes an HCI study of online design critique, so the paper is relevant prior work on critique. It reports 12 interview participants, not 26, and the retrieved passages do not establish that its study was AI-supported or that a sample of 26 aligns with its sample size. The citing authors’ statement that 26 exceeds their own power-analysis bound is not attributed to this source. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 57

  1. Bernadette Blair. 2006. Perception interpretation impact; an examination of the learning value of formative feedback to students through the design studio critique. Ph.D. Dissertation. Institute of Education, University of London.
  2. Benjamin M Bolker, Mollie E Brooks, Connie J Clark, Shane W Geange, John R Poulsen, M Henry H Stevens, and Jada-Simone S White. 2009. Generalized linear mixed models: a practical guide for ecology and evolution. Trends in Ecology & Evolution 24, 3 (2009), 127-135. https://doi.org/10.1016/j.tree.2008.10.008 · DOI · Paper page
  3. Moira Burke, Robert Kraut, and Elisabeth Joyce. 2010. Membership claims and requests: Conversation-level newcomer socialization strategies in online groups. Small group research 41, 1 (2010), 4-40.
  4. Deborah L Butler and Philip H Winne. 1995. Feedback and self-regulated learning: A theoretical synthesis. Review of educational research 65, 3 (1995), 245-281.
  5. Julia Cambre, Scott Klemmer, and Chinmay Kulkarni. 2018. Juxtapeer: Comparative Peer Review Yields Higher Quality Feedback and Promotes Deeper Reflection. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1-13. https://doi.org/10.1145/3173574.3173868 · DOI
  6. Julie Campbell, Cecilia Aragon, Katie Davis, Sarah Evans, Abigail Evans, and David Randall. 2016. Thousands of Positive Reviews: Distributed Mentoring in Online Fan Communities. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (San Francisco, California, USA) (CSCW '16). Association for Computing Machinery, New York, NY, USA, 691-704. https://doi.org/10.1145/2818048.2819934 · DOI
  7. J.M. Carroll. 2012. Creativity and Rationale: Enhancing Human Experience by Design. Springer London. https: //books.google.com/books?id=_dZJxlxK0NgC
  8. Allan Collins, John Seely Brown, and Ann Holum. 1991. Cognitive apprenticeship: Making thinking visible. American educator 15, 3 (1991), 6-11.
  9. Amy Cook, Jessica Hammer, Salma Elsayed-Ali, and Steven Dow. 2019. How Guiding Questions Facilitate Feedback Exchange in Project-Based Learning. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1-12. https: //doi.org/10.1145/3290605.3300368 · DOI
  10. Patrick A. Crain and Brian P. Bailey. 2017. Share Once or Share Often? Exploring How Designers Approach Iteration in a Large Online Community. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition (Singapore, Singapore) (C&C '17). Association for Computing Machinery, New York, NY, USA, 80-92. https://doi.org/10.1145/ 3059454.3059476 · DOI
  11. Deanna Dannels, Amy Housley Gaffney, and Kelly Norris Martin. 2008. Beyond Content, Deeper than Delivery: What Critique Feedback Reveals about Communication Expectations in Design Education. International Journal for the Scholarship of teaching and Learning 2, 2 (2008), n2.
  12. Jelle van Dijk, Jirka van der Roest, Remko van der Lugt, and Kees C.J. Overbeeke. 2011. NOOT: A Tool for Sharing Moments of Reflection during Creative Meetings. In Proceedings of the 8th ACM Conference on Creativity and Cognition (Atlanta, Georgia, USA) (C&C '11). Association for Computing Machinery, New York, NY, USA, 157-164. https: //doi.org/10.1145/2069618.2069646 · DOI
  13. Steven Dow, Julie Fortuna, Dan Schwartz, Beth Altringer, Daniel Schwartz, and Scott Klemmer. 2011. Prototyping Dynamics: Sharing Multiple Designs Improves Exploration, Group Rapport, and Results. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI '11). Association for Computing Machinery, New York, NY, USA, 2807-2816. https://doi.org/10.1145/1978942.1979359 · DOI · Paper page
  14. Steven Dow, Elizabeth Gerber, and Audris Wong. 2013. A Pilot Study of Using Crowds in the Classroom. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI '13). Association for Computing Machinery, New York, NY, USA, 227-236. https://doi.org/10.1145/2470654.2470686 · DOI
  15. Steven P. Dow, Alana Glassco, Jonathan Kass, Melissa Schwarz, Daniel L. Schwartz, and Scott R. Klemmer. 2011. Parallel Prototyping Leads to Better Design Results, More Divergence, and Increased Self-Efficacy. ACM Trans. Comput.-Hum. Interact. 17, 4, Article 18 (Dec. 2011), 24 pages. https://doi.org/10.1145/1879831.1879836 · DOI · Paper page
  16. Mukaddes Fasli and Badiossadat Hassanpour. 2017. Rotational critique system as a method of culture change in an architecture design studio: urban design studio as case study. Innovations in Education and Teaching International 54, 3 (2017), 194-205.
  17. Gerhard Fischer, Kumiyo Nakakoji, Jonathan Ostwald, Gerry Stahl, and Tamara Sumner. 1993. Embedding critics in design environments. The knowledge engineering review 8, 4 (1993), 285-307.
  18. David Fletcher. 2008. Confidence intervals for the mean of the delta-lognormal distribution. Environmental and Ecological Statistics 15, 2 (2008), 175-189.
  19. Eureka Foong, Steven P. Dow, Brian P. Bailey, and Elizabeth M. Gerber. 2017. Online Feedback Exchange: A Framework for Understanding the Socio-Psychological Factors. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). Association for Computing Machinery, New York, NY, USA, 4454-4467. https://doi.org/10.1145/3025453.3025791 · DOI
  20. Denae Ford, Kristina Lustig, Jeremy Banks, and Chris Parnin. 2018. "We Don't Do That Here": How Collaborative Editing with Mentors Improves Engagement in Social Q&A Communities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1-12. https://doi.org/10.1145/3173574.3174182 · DOI
  21. Sharon Nelson-Le Gall. 1985. Chapter 2: Help-Seeking Behavior in Learning. Review of Research in Education 12, 1 (1985), 55-90.
  22. Elizabeth M Gerber, Jeanne Marie Olson, and Rebecca LD Komarek. 2012. Extracurricular design-based learning: Preparing students for careers in innovation. International Journal of Engineering Education 28, 2 (2012), 317.
  23. F Maxwell Harper, Joseph Weinberg, John Logie, and Joseph A Konstan. 2010. Question types in social Q&A sites. First Monday 15, 7 (2010).
  24. John Hattie and Helen Timperley. 2007. The power of feedback. Review of educational research 77, 1 (2007), 81-112.
  25. Catherine M. Hicks, Vineet Pandey, C. Ailie Fraser, and Scott Klemmer. 2016. Framing Feedback: Choosing Review Environment Features That Support High Quality Peer Assessment. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). Association for Computing Machinery, New York, NY, USA, 458-469. https://doi.org/10.1145/2858036.2858195 · DOI
  26. John Horner and Michael E Atwood. 2006. Effective design rationale: understanding the barriers. In Rationale management in software engineering. Springer, 73-90.
  27. Julie S. Hui, Amos Glenn, Rachel Jue, Elizabeth M Gerber, and Steven P. Dow. 2015. Using Anonymity and Communal Efforts to Improve Quality of Crowdsourced Feedback. In Proceedings of the Third AAAI Conference on Human Computation and Crowdsourcing, Elizabeth Gerber and Panos Ipeirotis (Eds.). AAAI Press, 72-82.
  28. Julie S Hui, Matthew W Easterday, and Elizabeth M Gerber. 2019. Distributed Apprenticeship in Online Communities. Human-Computer Interaction 34, 4 (2019), 328-378.
  29. Julie S. Hui, Elizabeth M. Gerber, and Steven P. Dow. 2014. Crowd-Based Design Activities: Helping Students Connect with Users Online. In Proceedings of the 2014 Conference on Designing Interactive Systems (Vancouver, BC, Canada) (DIS '14). Association for Computing Machinery, New York, NY, USA, 875-884. https://doi.org/10.1145/2598510.2598538 · DOI
  30. Julie S. Hui, Darren Gergle, and Elizabeth M. Gerber. 2018. IntroAssist: A Tool to Support Writing Introductory Help Requests. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1-13. https://doi.org/10.1145/3173574.3173596 · DOI · Paper page
  31. Joy Kim, Maneesh Agrawala, and Michael S. Bernstein. 2017. Mosaic: Designing Online Creative Communities for Sharing Works-in-Progress. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17). Association for Computing Machinery, New York, NY, USA, 246-258. https://doi.org/10.1145/2998181.2998195 · DOI
  32. Yasmine Kotturi and McKayla Kingston. 2019. Why Do Designers in the "Wild" Wait to Seek Feedback until Later in Their Design Process?. In Proceedings of the 2019 on Creativity and Cognition (San Diego, CA, USA) (C&C '19). Association for Computing Machinery, New York, NY, USA, 541-546. https://doi.org/10.1145/3325480.3326580 · DOI
  33. Yubo Kou and Colin M Gray. 2017. Supporting distributed critique through interpretation and sense-making in an online creative community. Proceedings of the ACM on Human-Computer Interaction 1, CSCW (2017), 60.
  34. Yubo Kou and Colin M. Gray. 2018. "What Do You Recommend a Complete Beginner like Me to Practice?": Professional Self-Disclosure in an Online Community. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 94 (Nov. 2018), 24 pages. https://doi.org/10.1145/3274363 · DOI
  35. Markus Krause, Tom Garncarz, JiaoJiao Song, Elizabeth M. Gerber, Brian P. Bailey, and Steven P. Dow. 2017. Critique Style Guide: Improving Crowdsourced Design Feedback with a Natural Language Model. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). Association for Computing Machinery, New York, NY, USA, 4627-4639. https://doi.org/10.1145/3025453.3025883 · DOI
  36. Robert E Kraut and Paul Resnick. 2012. Building successful online communities: Evidence-based social design. Mit Press.
  37. Chinmay E. Kulkarni, Michael S. Bernstein, and Scott R. Klemmer. 2015. PeerStudio: Rapid Peer Feedback Emphasizes Revision and Improves Performance. In Proceedings of the Second (2015) ACM Conference on Learning @ Scale (Vancouver, BC, Canada) (L@S '15). Association for Computing Machinery, New York, NY, USA, 75-84. https://doi.org/10.1145/ 2724660.2724670 · DOI · Paper page
  38. Kurt Luther, Jari-Lee Tolentino, Wei Wu, Amy Pavel, Brian P. Bailey, Maneesh Agrawala, Björn Hartmann, and Steven P. Dow. 2015. Structuring, Aggregating, and Evaluating Crowdsourced Design Critique. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (Vancouver, BC, Canada) (CSCW '15). Association for Computing Machinery, New York, NY, USA, 473-485. https://doi.org/10.1145/2675133.2675283 · DOI
  39. Jennifer Marlow and Laura Dabbish. 2014. From Rookie to All-Star: Professional Development in a Graphic Design Social Networking Site. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (Baltimore, Maryland, USA) (CSCW '14). Association for Computing Machinery, New York, NY, USA, 922-933. https://doi.org/10.1145/2531602.2531651 · DOI
  40. Tanushree Mitra and Eric Gilbert. 2014. The Language That Gets People to Give: Phrases That Predict Success on Kickstarter. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (Baltimore, Maryland, USA) (CSCW '14). Association for Computing Machinery, New York, NY, USA, 49-61. https: //doi.org/10.1145/2531602.2531656 · DOI
  41. Kumiyo Nakakoji, Yasuhiro Yamamoto, Shingo Takada, and Brent N. Reeves. 2000. Two-Dimensional Spatial Positioning as a Means for Reflection in Design. In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (New York City, New York, USA) (DIS '00). Association for Computing Machinery, New York, NY, USA, 145-154. https://doi.org/10.1145/347642.347697 · DOI
  42. Tricia J Ngoon, C Ailie Fraser, Ariel S Weingarten, Mira Dontcheva, and Scott Klemmer. 2018. Interactive Guidance Techniques for Improving Creative Feedback. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 55.
  43. David J Nicol and Debra Macfarlane-Dick. 2006. Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in higher education 31, 2 (2006), 199-218.
  44. Minna Puustinen and Jean-François Rouet. 2009. Learning with new technologies: Help seeking and information searching revisited. Computers & Education 53, 4 (2009), 1014 -1019. https://doi.org/10.1016/j.compedu.2008.07.002 Learning with ICT: New perspectives on help seeking and information searching. · DOI
  45. Ken Reily, Pam Ludford Finnerty, and Loren Terveen. 2009. Two Peers Are Better than One: Aggregating Peer Reviews for Computing Assignments is Surprisingly Accurate. In Proceedings of the ACM 2009 International Conference on Supporting Group Work (Sanibel Island, Florida, USA) (GROUP '09). Association for Computing Machinery, New York, NY, USA, 115-124. https://doi.org/10.1145/1531674.1531692 · DOI
  46. Brian J Reiser. 2004. Scaffolding complex learning: The mechanisms of structuring and problematizing student work. The Journal of the Learning sciences 13, 3 (2004), 273-304.
  47. Donald A Schön. 2017. The reflective practitioner: How professionals think in action. Routledge.
  48. Amy Shannon, Jessica Hammer, Hassler Thurston, Natalie Diehl, and Steven Dow. 2016. PeerPresents: A Web-Based System for In-Class Peer Feedback during Student Presentations. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (Brisbane, QLD, Australia) (DIS '16). Association for Computing Machinery, New York, NY, USA, 447-458. https://doi.org/10.1145/2901790.2901816 · DOI
  49. Oscar Tomico, Joep W. Frens, and C. J. Overbeeke. 2009. Co-Reflection: User Involvement for Highly Dynamic Design Processes. In CHI '09 Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA) (CHI EA '09). Association for Computing Machinery, New York, NY, USA, 2695-2698. https://doi.org/10.1145/1520340.1520389 · DOI
  50. Anbang Xu and Brian Bailey. 2012. What Do You Think? A Case Study of Benefit, Expectation, and Interaction in a Large Online Critique Community. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (Seattle, Washington, USA) (CSCW '12). Association for Computing Machinery, New York, NY, USA, 295-304. https://doi.org/10.1145/2145204.2145252 · DOI
  51. Anbang Xu, Shih-Wen Huang, and Brian Bailey. 2014. Voyant: Generating Structured Feedback on Visual Designs Using a Crowd of Non-Experts. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (Baltimore, Maryland, USA) (CSCW '14). Association for Computing Machinery, New York, NY, USA, 1433-1444. https://doi.org/10.1145/2531602.2531604 · DOI
  52. Lichun Yang, Shenghua Bao, Qingliang Lin, Xian Wu, Dingyi Han, Zhong Su, and Yong Yu. 2011. Analyzing and Predicting Not-Answered Questions in Community-Based Question Answering Services. In Proceedings of the Twenty- Fifth AAAI Conference on Artificial Intelligence (San Francisco, California) (AAAI'11). AAAI Press, 1273-1278.
  53. Lichun Yang, Shenghua Bao, Qingliang Lin, Xian Wu, Dingyi Han, Zhong Su, and Yong Yu. 2011. Analyzing and Predicting Not-Answered Questions in Community-Based Question Answering Services. In Proceedings of the Twenty- Fifth AAAI Conference on Artificial Intelligence (San Francisco, California) (AAAI'11). AAAI Press, 1273-1278.
  54. Yu Chun Grace Yen, Steven P. Dow, Elizabeth Gerber, and Brian P. Bailey. 2017. Listen to others, listen to yourself: Combining feedback review and reflection to improve iterative design. In C and C 2017 -Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition (C and C 2017 -Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition). Association for Computing Machinery, Inc, 158-170. https://doi.org/10.1145/3059454.3059468 · DOI
  55. Yu-Chun (Grace) Yen, Steven P. Dow, Elizabeth Gerber, and Brian P. Bailey. 2016. Social Network, Web Forum, or Task Market? Comparing Different Crowd Genres for Design Feedback Exchange. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (Brisbane, QLD, Australia) (DIS '16). Association for Computing Machinery, New York, NY, USA, 773-784. https://doi.org/10.1145/2901790.2901820 · DOI
  56. Yu-Chun Grace Yen, Joy O. Kim, and Brian P. Bailey. 2020. Decipher: An Interactive Visualization Tool for Interpreting Unstructured Design Feedback from Multiple Providers. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1-13. https://doi.org/10.1145/3313831.3376380 · DOI
  57. Alvin Yuan, Kurt Luther, Markus Krause, Sophie Isabel Vennix, Steven P Dow, and Bjorn Hartmann. 2016. Almost an Expert: The Effects of Rubrics and Expertise on Perceived Value of Crowdsourced Design Critiques. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (San Francisco, California, USA) (CSCW '16). Association for Computing Machinery, New York, NY, USA, 1005-1017. https://doi.org/10.1145/ 2818048.2819953 · DOI

Cited by

Sorted by citations

Authors
Cheng, R., Zeng, Z., Liu, M., & Dow, S.
Affiliations
  • University of Washington (US)
  • University of California San Diego (US)
  • University of California, Davis (US)
ORCID
Venue
Proceedings of the ACM on Human-Computer Interaction
Publisher
Association for Computing Machinery (ACM)
Published
2020-10-14
Volume
4
Issue
CSCW2
Pages
1-24
ISSN
2573-0142 (ISSN-L)
DOI
10.1145/3415232
OpenAlex
W3093641007
Cited by
33
References
2
Citations by year
2026: 4 · 2025: 6 · 2024: 6 · 2023: 3 · 2022: 8 · 2021: 6
FWCI
4.00
Type
journal article
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
Funders
NSF (#1821618, #1122320 and #1821618)
Topics
  • Physical Sciences › Computer Science › Information Systems › Expert finding and Q&A systems
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
Keywords
feedback seeking, feedback exchange, request strategies, design criticism, self-criticism, design variants, feedback quality, online communities, creative work

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review