Improving Crowd Innovation with Expert Facilitation
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Online crowds are a promising source of new innovations. However, crowd innovation quality does not always match its quantity. In this paper, we explore how to improve crowd innovation with real-time expert guidance. One approach would for experts to provide personalized feed-back, but this scales poorly, and may lead to premature convergence during creative work. Drawing on strategies for facilitating face-to-face brainstorms, we introduce a crowd ideation system where experts monitor incoming ideas through a dashboard and offer high-level “inspirations” to guide ideation. A series of controlled experiments show that experienced facilitators increased the quantity and creativity of workers' ideas compared to unfacilitated workers, while Novice facilitators reduced workers' creativity. Analyses of inspiration strategies suggest these opposing results stem from differential use of successful inspiration strategies (e.g., provoking mental simulations). The results show that expert facilitation can significantly improve crowd innovation, but inexperienced facilitators may need scaffolding to be successful.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087
An introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880 · cited 4×
“Breadth and Depth of Search of Solution Space To characterize the structure of the solution space, we trained a semantic model of the set of ideas using Latent Semantic Analysis (LSA [30]), which estimates a highdimensional semantic space representation of a corpus of documents based on word co-occurrence patterns.”
MethodInfluentialConducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Hennessey, B. A., & Amabile, T. M. (2010) · Annual Review of Psychology · doi:10.1146/annurev.psych.093008.100416 · Cited by 1,475
Mechanization in problem solving: The effect of Einstellung.
Luchins, A. S. (1942) · Psychological Monographs · doi:10.1037/h0093502 · Cited by 1,143
Social facilitation: A meta-analysis of 241 studies.
Bond, C. F., & Titus, L. J. (1983) · Psychological Bulletin · doi:10.1037/0033-2909.94.2.265 · Cited by 684 · cited 1×
“First, the benefits of facilitation cannot be explained in terms of social facilitation [6], i.”
BackgroundPersonas and user-centered design: how can personas benefit product design processes?
Miaskiewicz, T., & Kozar, K. A. (2011) · Design Studies · doi:10.1016/j.destud.2011.03.003 · Cited by 559
Peer and self assessment in massive online classes
Kulkarni, C., Wei, K. P., Le, H., Chia, D., Papadopoulos, K., Cheng, J., Koller, D., & Klemmer, S. R. (2013) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2505057 · Cited by 390 · cited 1×
“For example, peer-review systems have successfully employed scaffolding techniques, such as preauthored templates to improve feedback at scale [27,34].”
MethodShepherding the crowd yields better work
Dow, S., Kulkarni, A., Klemmer, S., & Hartmann, B. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145355 · Cited by 334 · cited 3×
“Our research builds on techniques designed to enhance quality on more open-ended crowd tasks by leveraging experts for real-time input [14,22].”
BackgroundMethodSelf-citationProviding Timely Examples Improves the Quantity and Quality of Generated Ideas
Siangliulue, P., Chan, J., Gajos, K. Z., & Dow, S. P. (2015) · Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition · doi:10.1145/2757226.2757230 · Cited by 99 · cited 1×
“This “pull” approach supports greater scalability and was motivated by prior work showing that ideators benefit most from inspirations when delivered “on demand” versus pushed or on a regular interval [49].”
BackgroundSelf-citation
Cited by
Mapping the Landscape of Creativity Support Tools in HCI
Frich, J., MacDonald Vermeulen, L., Remy, C., Biskjaer, M. M., & Dalsgaard, P. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300619 · Cited by 410
“2018 [29, 30, 62, 73, 90, 111, 126, 162, 165, 171, 175, 184] 2017 [3, 7, 36, 43, 64, 82, 94, 98, 102, 118, 130, 134, 150, 158, 176, 183, 192] 2016 [9, 25, 35, 39, 57, 63, 66, 79, 81, 87, 101, 114, 117, 127, 129, 159, 167, 187] 2015 [38, 54, 89, 100, 123, 137, 160, 166, 168, 186] 2014 [14, 37, 55, 71, 83, 88, 97, 99, 1…”
Shen, H., DeVos, A., Eslami, M., & Holstein, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479577 · Cited by 171
“To help steer everyday auditors’ ongoing efforts in more productive directions, designers might invite relevant experts to continuously monitor community discussions and intervene as needed [14, 27].”
BackgroundKim, S., Eun, J., Oh, C., Suh, B., & Lee, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376785 · Cited by 133
Evaluating Creativity Support Tools in HCI Research
Remy, C., MacDonald Vermeulen, L., Frich, J., Biskjaer, M. M., & Dalsgaard, P. (2020) · Proceedings of the 2020 ACM Designing Interactive Systems Conference · doi:10.1145/3357236.3395474 · Cited by 96
“2018 [105, 142, 150, 30, 57, 29, 85, 115, 146, 66] 2017 [90, 5, 2, 152, 45, 164, 58, 79, 123, 120, 136, 99, 109, 36, 95, 139] 2016 [119, 59, 160, 78, 82, 107, 143, 76, 39, 98, 6, 108, 25, 35, 121] 2015 [51, 38, 159, 126, 140, 144, 97, 84, 112] 2014 [37, 156, 111, 96, 113, 163, 155, 52, 80, 94, 13, 83] 2013 [158, 81, 1…”
Ding, Z., Srinivasan, A., Macneil, S., & Chan, J. (2023) · Creativity and Cognition · doi:10.1145/3591196.3593516 · Cited by 35
“Another interesting direction may be to leverage LLM-generated analogies to improve the process of iteration from feedback [11, 12, 22].”
BackgroundSelf-citation
Metadata
- Authors
- Chan, J., Dang, S., & Dow, S. P.
- Venue
- Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing
- Publisher
- ACM
- Published
- 2016-02-27
- Pages
- 1223-1235
- OpenAlex
- W2290102723
- Cited by
- 86
- References
- 10
- FWCI
- 38.08
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Improving Crowd Innovation with Expert Facilitation. https://app.cochat.ai/papers/10.1145/2818048.2820023
- The paper
- Chan, J., Dang, S., & Dow, S. P. (2016). Improving Crowd Innovation with Expert Facilitation. Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. https://doi.org/10.1145/2818048.2820023
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.