Providing Timely Examples Improves the Quantity and Quality of Generated Ideas
Abstract
Emerging online ideation platforms with thousands of example ideas provide an important resource for creative production. But how can ideators best use these examples to create new innovations? Recent work has suggested that not just the choice of examples, but also the timing of their delivery can impact creative outcomes. Building on existing cognitive theories of creative insight, we hypothesize that people are likely to benefit from examples when they run out of ideas. We explore two example delivery mechanisms that test this hypothesis: 1) a system that proactively provides examples when a user appears to have run out of ideas, and 2) a system that provides examples when a user explicitly requests them. Our online experiment (N=97) compared these two mechanisms against two baselines: providing no examples and automatically showing examples at a regular interval. Participants who requested examples themselves generated ideas that were rated the most novel by external evaluators. Participants who received ideas automatically when they appeared to be stuck produced the most ideas. Importantly, participants who received examples at a regular interval generated fewer ideas than participants who received no examples, suggesting that mere access to examples is not sufficient for creative inspiration. These results emphasize the importance of the timing of example delivery. Insights from this study can inform the design of collective ideation support systems that help people generate many high quality ideas.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Users exposed to regularly-spaced examples produced the fewest ideas.
“Similarly, Siangliulue et al. investigated the efect of when examples were shown; users exposed to regularly-spaced examples produced the fewest ideas [44].”
From Assessing the Impact of Automated Suggestions on Decision Making: Domain Experts Mediate Model Errors but Take Less Initiative (2021) · cites this paper as [44]
Cites
Productivity loss in brainstorming groups: Toward the solution of a riddle.
Diehl, M., & Stroebe, W. (1987) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.53.3.497 · Cited by 1,925 · see citing works
Flow. The Psychology of Optimal Experience. New York (HarperPerennial) 1990.
Csíkszentmihályi, M. (1990) · Cited by 1,864 · see citing works
The dual pathway to creativity model: Creative ideation as a function of flexibility and persistence
Nijstad, B. A., Dreu, C. K. W. D., Rietzschel, E. F., & Baas, M. (2010) · European Review of Social Psychology · doi:10.1080/10463281003765323 · Cited by 1,062 · see citing works
Kapur, M. (2008) · Cognition and Instruction · doi:10.1080/07370000802212669 · Cited by 867 · see citing works
Structured Imagination: the Role of Category Structure in Exemplar Generation
Ward, T. B. (1994) · Cognitive Psychology · doi:10.1006/cogp.1994.1010 · Cited by 828 · see citing works
Schwartz, D. L., & Martin, T. (2004) · Cognition and Instruction · doi:10.1207/s1532690xci2202_1 · Cited by 763 · see citing works
The Influence and Value of Analogical Thinking during New Product Ideation
Dahl, D. W., & Moreau, P. (2002) · Journal of Marketing Research · doi:10.1509/jmkr.39.1.47.18930 · Cited by 630 · see citing works
Reasoning in humans. II. The solution of a problem and its appearance in consciousness
Maier, N. R. F. (1931) · Journal of Comparative Psychology · doi:10.1037/h0071361 · Cited by 621 · see citing works
Design and other types of fixation
Purcell, A. T., & Gero, J. S. (1996) · Design Studies · doi:10.1016/s0142-694x(96)00023-3 · Cited by 552 · see citing works
Social influence processes in group brainstorming.
Paulus, P. B., & Dzindolet, M. T. (1993) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.64.4.575 · Cited by 473 · see citing works
Schwartz, D. L., Chase, C. C., Oppezzo, M., & Chin, D. B. (2011) · Journal of Educational Psychology · doi:10.1037/a0025140 · Cited by 466 · see citing works
A Study of Design Fixation, Its Mitigation and Perception in Engineering Design Faculty
Linsey, J., Tseng, I.-H., Fu, K. K., Cagan, J., Wood, K. L., & Schunn, C. D. (2010) · Journal of Mechanical Design · doi:10.1115/1.4001110 · Cited by 393 · see citing works
Chrysikou, E. G., & Weisberg, R. W. (2005) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.31.5.1134 · Cited by 346 · see citing works
Cognitive stimulation and interference in groups: Exposure effects in an idea generation task
Nijstad, B. A., Stroebe, W., & Lodewijkx, H. F. M. (2002) · Journal of Experimental Social Psychology · doi:10.1016/s0022-1031(02)00500-0 · Cited by 342 · see citing works
Rietzschel, E. F., Nijstad, B. A., & Stroebe, W. (2006) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2006.10.014 · Cited by 333 · see citing works
Bailey, B. P., & Iqbal, S. T. (2008) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1314683.1314689 · Cited by 325 · see citing works
Chan, J., Fu, K. K., Schunn, C. D., Cagan, J., Wood, K. L., & Kotovsky, K. (2011) · Journal of Mechanical Design · doi:10.1115/1.4004396 · Cited by 293 · see citing works
Self-citationCognitive and social comparison processes in brainstorming
Dugosh, K. L., & Paulus, P. B. (2004) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2004.05.009 · Cited by 292 · see citing works
Attentional Priming Effects on Creativity
Friedman, R., Ayelet Fishbach, B., Förster, J., & Werth, L. (2003) · Creativity Research Journal · doi:10.1080/10400419.2003.9651420 · Cited by 275 · see citing works
Spontaneous analogising in engineering design: a comparative analysis of experts and novices
Ball, L. J., Ormerod, T. C., & Morley, N. (2004) · Design Studies · doi:10.1016/j.destud.2004.05.004 · Cited by 272 · see citing works
Herring, S. R., Chang, C., Krantzler, J., & Bailey, B. P. (2009) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518717 · Cited by 261 · see citing works
Demystification of Cognitive Insight: Opportunistic Assimilation and the Prepared-Mind Perspective
Seifert, C. M., Meyer, D. E., Davidson, N., Patalano, A. L., & Yaniv, I. (1994) · The MIT Press eBooks · doi:10.7551/mitpress/4879.003.0007 · Cited by 257 · see citing works
Rietzschel, E. F., Nijstad, B. A., & Stroebe, W. (2007) · PsycEXTRA Dataset · doi:10.1037/e633982013-064 · Cited by 237 · see citing works
Why good thoughts block better ones: The mechanism of the pernicious Einstellung (set) effect
Bilalić, M., McLeod, P., & Gobet, F. R. (2008) · Cognition · doi:10.1016/j.cognition.2008.05.005 · Cited by 236 · see citing works
Invoking Social Comparison to Improve Electronic Brainstorming: Beyond Anonymity
Shepherd, M. M., Briggs, R. O., Reinig, B. A., Yen, J., & Nunamaker, J. F. (1995) · Journal of Management Information Systems · doi:10.1080/07421222.1995.11518095 · Cited by 225 · see citing works
How examples may (and may not) constrain creativity
Marsh, R. L., Landau, J. D., & Hicks, J. L. (1996) · Memory & Cognition · doi:10.3758/bf03201091 · Cited by 206 · see citing works
Functional Fixedness in a Technologically Sparse Culture
German, T. P., & Barrett, H. C. (2005) · Psychological Science · doi:10.1111/j.0956-7976.2005.00771.x · Cited by 205 · see citing works
The role of timing and analogical similarity in the stimulation of idea generation in design
Tseng, I., Moss, J., Cagan, J., & Kotovsky, K. (2008) · Design Studies · doi:10.1016/j.destud.2008.01.003 · Cited by 204 · see citing works
The Impact of Type of Examples on Originality: Explaining Fixation and Stimulation Effects
Agogué, M., Kazakci, A. O., Hatchuel, A., Masson, P. L., Weil, B., Poirel, N., & Cassotti, M. (2013) · The Journal of Creative Behavior · doi:10.1002/jocb.37 · Cited by 195 · see citing works
Michinov, N., & Primois, C. (2004) · Computers in Human Behavior · doi:10.1016/j.chb.2004.02.004 · Cited by 186 · see citing works
Effects of extended effort in creative problem solving.
Parnes, S. J. (1961) · Journal of Educational Psychology · doi:10.1037/h0044650 · Cited by 182 · see citing works
Designing with interactive example galleries
Lee, B., Srivastava, S., Kumar, R., Brafman, R., & Klemmer, S. R. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753667 · Cited by 178 · cited 2× · see citing works
“Ideas generated by others can help innovators working on similar problems spur new concepts by broadening their notion of the design space [19, 24, 40] and allowing for reinterpretation and recombination of ideas [19, 50, 28].”
BackgroundYu, L., & Nickerson, J. V. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979147 · Cited by 169 · see citing works
Motivation gains of inferior group members: A meta-analytical review.
Weber, B., & Hertel, G. (2007) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.93.6.973 · Cited by 162 · see citing works
Kumar, R., Satyanarayan, A., Torres, C., Lim, M., Ahmad, S., Klemmer, S., & Talton, J. O. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466420 · Cited by 141 · see citing works
Contributions of inadequate source monitoring to unconscious plagiarism during idea generation.
Marsh, R. L., Landau, J. D., & Hicks, J. L. (1997) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.23.4.886 · Cited by 137 · see citing works
Toward Collaborative Ideation at Scale
Siangliulue, P., Arnold, K. C., Gajos, K. Z., & Dow, S. P. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675239 · Cited by 127 · see citing works
Self-citationMeasuring the effects of interruptions on task performance in the user interface
Bailey, B. P., Konstan, J. A., & Carlis, J. V. (2002) · SMC 2000 Conference Proceedings. 2000 IEEE International Conference on Systems, Man and Cybernetics. 'Cybernetics Evolving to Systems, Humans, Organizations, and their Complex Interactions' (Cat. No.00CH37166) · doi:10.1109/icsmc.2000.885940 · Cited by 127 · see citing works
Some Liberating Effects of Anonymous Electronic Brainstorming
Cooper, W. H., Gallupe, R. B., Pollard, S., & Cadsby, J. (1998) · Small Group Research · doi:10.1177/1046496498292001 · Cited by 120 · see citing works
The influence of open goals on the acquisition of problem-relevant information.
Moss, J., Kotovsky, K., & Cagan, J. (2007) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.33.5.876 · Cited by 80 · see citing works
Category assignment and relatedness in the group ideation process
Baruah, J., & Paulus, P. B. (2011) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2011.04.007 · Cited by 79 · see citing works
Ritchie, D., Kejriwal, A. A., & Klemmer, S. (2011) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2047196.2047216 · Cited by 73 · see citing works
Early and Repeated Exposure to Examples Improves Creative Work
Kulkarni, C., Dow, S. P., & Klemmer, S. R. (2014) · Design Thinking Research · doi:10.1007/978-3-319-01303-9_4 · Cited by 61 · see citing works
Self-citationEckert, C. M., & Stacey, M. (1998) · Creativity and Innovation Management · doi:10.1111/1467-8691.00080 · Cited by 41 · see citing works
Memory for impasses during problem solving
Patalano, A. L., & Seifert, C. M. (1994) · Memory & Cognition · doi:10.3758/bf03208894 · Cited by 38 · see citing works
Doboli, A., Umbarkar, A., Subramanian, V., & Doboli, S. (2013) · Design Studies · doi:10.1016/j.destud.2013.10.002 · Cited by 32 · see citing works
Zeng, L., Proctor, R. W., & Salvendy, G. (2011) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811409219 · Cited by 27 · see citing works
Xu, A., & Bailey, B. P. (2012) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2145204.2145380 · Cited by 20 · see citing works
An Exploration of Real-World Analogical Problem Solving in Novices
Bearman, C. R., Ball, L. J., & Ormerod, T. C. (2019) · Routledge eBooks · doi:10.4324/9781315782379-55 · Cited by 16 · see citing works
Chan, J., Dow, S. P., & Schunn, C. D. (2014) · doi:10.1145/2556420.2556500 · Cited by 12 · see citing works
Self-citationCrowd creativity through combination
Yu, L. (2011) · Creativity and Cognition · doi:10.1145/2069618.2069756 · Cited by 11 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Mapping the Landscape of Creativity Support Tools in HCI
Frich, J., MacDonald Vermeulen, L., Remy, C., Biskjaer, M. M., & Dalsgaard, P. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300619 · Cited by 410 · see citing works
Evaluating Creativity Support Tools in HCI Research
Remy, C., MacDonald Vermeulen, L., Frich, J., Biskjaer, M. M., & Dalsgaard, P. (2020) · Proceedings of the 2020 ACM Designing Interactive Systems Conference · doi:10.1145/3357236.3395474 · Cited by 96 · see citing works
Levy, A., Agrawal, M., Satyanarayan, A., & Sontag, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445522 · Cited by 73 · see citing works
Ding, Z., Srinivasan, A., Macneil, S., & Chan, J. (2023) · Creativity and Cognition · doi:10.1145/3591196.3593516 · Cited by 35 · see citing works
Self-citationDesign Fixation From Initial Examples: Provided Versus Self-Generated Ideas
Leahy, K., Daly, S. R., McKilligan, S., & Seifert, C. M. (2020) · Journal of Mechanical Design · doi:10.1115/1.4046446 · Cited by 33 · see citing works
IdeateRelate: An Examples Gallery That Helps Creators Explore Ideas in Relation to Their Own
Xu, X. (., Xiong, R., Wang, B., Min, D., & Dow, S. P. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479496 · Cited by 21 · see citing works
Self-citationMapping the Design Space of Interactions in Human-AI Text Co-creation Tasks
Ding, Z., & Chan, J. (2023) · arXiv · doi:10.48550/arxiv.2303.06430 · Cited by 9 · see citing works
Self-citation
- Authors
- Siangliulue, P., Chan, J., Gajos, K. Z., & Dow, S. P.
- Affiliations
- Harvard University (US)
- Carnegie Mellon University (US)
- Venue
- Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition
- Publisher
- ACM
- Published
- 2015-06-22
- Pages
- 83-92
- OpenAlex
- W2144569346
- Cited by
- 99
- References
- 51
- Citations by year
- 2026: 4 · 2025: 11 · 2024: 8 · 2023: 5 · 2022: 10 · 2021: 12 · 2020: 9 · 2019: 10 · 2018: 6 · 2017: 12 · 2016: 9 · 2015: 3
- FWCI
- 26.93
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Creativity and Cognition (publishedVersion)
- Funders
- National Science Foundation (IIS-1208382, IIS-1217096)
- Adobe Systems
- Topics
- Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Creativity in Education and Neuroscience
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- idea generation, creative insight, idea novelty, artistic inspiration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
