Parallel prototyping leads to better design results, more divergence, and increased self-efficacy
The DOI matches its Crossref record; 67 of the 74 references it cites were found in a registry or an index.
Abstract
Iteration can help people improve ideas. It can also give rise to fixation, continuously refining one option without considering others. Does creating and receiving feedback on multiple prototypes in parallel, as opposed to serially, affect learning, self-efficacy, and design exploration? An experiment manipulated whether independent novice designers created graphic Web advertisements in parallel or in series. Serial participants received descriptive critique directly after each prototype. Parallel participants created multiple prototypes before receiving feedback. As measured by click-through data and expert ratings, ads created in the Parallel condition significantly outperformed those from the Serial condition. Moreover, independent raters found Parallel prototypes to be more diverse. Parallel participants also reported a larger increase in task-specific self-confidence. This article outlines a theoretical foundation for why parallel prototyping produces better design results and discusses the implications for design education.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, these interfaces do not allow endusers to prototype confgurations in parallel [24], which can create hurdles for comparing the efects of diferent confgurations or combining aspects of the confgurations for further iteration and experimentation.”
From Cells, Generators, and Lenses: Design Framework for Object-Oriented Interaction with Large Language Models (2023) · cites this paper as [24]
In the citing paper
“our framework describes how interfaces can support end-users to fexibly assemble and reassemble these objects into diverse concurrent confgurations-supporting parallel prototyping [24] and mix-and-match between confgurations.”
From Cells, Generators, and Lenses: Design Framework for Object-Oriented Interaction with Large Language Models (2023) · cites this paper as [24]
In the citing paper
“Without first exploring diverse ideas, one can hastily converge to a sub-optimal idea and spend time iterating on it without considering other options - a phenomenon called fixation that hinders creative processes [19, 24, 37].”
From Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation (2024) · cites this paper as [24]
In the citing paper
“They found that divergent ideation promoted comparison and exploration, and led to better results and self-efficacy [24].”
From Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation (2024) · cites this paper as [24]
In the citing paper
“Second, the literature on creativity consistently underscores our tendency to prematurely converge [19, 24, 36, 37], unless interventions are employed, such as explicitly requiring the generation of multiple ideas at the start of the creative process [24, 54].”
From Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation (2024) · cites this paper as [24]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Self-efficacy: The exercise of control
Bandura, A., Freeman, W. H., & Lightsey, R. (1999) · Journal of Cognitive Psychotherapy · doi:10.1891/0889-8391.13.2.158 · Cited by 46,835
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Fredrickson, B. L. (2001) · American Psychologist · doi:10.1037//0003-066x.56.3.218 · Cited by 18,025
Dilemmas in a general theory of planning
Rittel, H. W. J., & Webber, M. M. (1973) · Policy Sciences · doi:10.1007/bf01405730 · Cited by 16,338
Flow: the psychology of optimal experience
(1990) · Choice Reviews Online · doi:10.5860/choice.28-0597 · Cited by 12,637
Flow: The Psychology of Optimal Experience
Buchanan, R., & Csikszentmihalyi, M. (1991) · Design Issues · doi:10.2307/1511458 · Cited by 9,394
Mindset: The New Psychology of Success
(2006) · Choice Reviews Online · doi:10.5860/choice.44-2397 · Cited by 7,804
(1987) · Cited by 6,910
A Pattern Language: Towns, Buildings, Construction
Alexander, C., Ishikawa, S., Silverstein, M., Jacobson, M., Fiksdahl-King, I., & Angel, S. (1977) · Cited by 6,779
Newell, A. (1972) · Cited by 6,222
The sciences of the artificial (3rd ed.)
Simon, H. A. (1996) · The MIT Press eBooks · Cited by 4,300
Kolodner, J. L. (2005) · The Cambridge Handbook of the Learning Sciences · doi:10.1017/cbo9780511816833.015 · Cited by 4,247
The Reflective Practitioner: How Professionals Think in Action
Schon, D. A., & DeSanctis, V. (1986) · The Journal of Continuing Higher Education · doi:10.1080/07377366.1986.10401080 · Cited by 3,224
Schema induction and analogical transfer
Gick, M. L., & Holyoak, K. J. (1983) · Cognitive Psychology · doi:10.1016/0010-0285(83)90002-6 · Cited by 3,010
The Psychology Of Intelligence
Piaget, J. (2005) · Routledge eBooks · doi:10.4324/9780203981528 · Cited by 3,003
Duncker, K. (1945) · Psychological Monographs · doi:10.1037/h0093599 · Cited by 2,938
Engineering Design Thinking, Teaching, and Learning
Dym, C. L., Agogino, A. M., Eris, O., Frey, D. D., & Leifer, L. J. (2005) · Journal of Engineering Education · doi:10.1002/j.2168-9830.2005.tb00832.x · Cited by 2,801
(1982) · Houghton Mifflin eBooks · Cited by 2,650
Arkes, H. R., & Blumer, C. (1985) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(85)90049-4 · Cited by 2,573
Creativity in the design process: co-evolution of problem–solution
Dorst, K., & Cross, N. (2001) · Design Studies · doi:10.1016/s0142-694x(01)00009-6 · Cited by 2,357
The reflective practitioner: How professionals think in action
Bogumil, R. J. (1985) · Proceedings of the IEEE · doi:10.1109/proc.1985.13210 · Cited by 2,323
The Psychology of Intelligence.
Strauss, A., Piaget, J., Piercy, M., & Berlyne, D. E. (1951) · American Sociological Review · doi:10.2307/2088298 · Cited by 2,073
Structure mapping in analogy and similarity.
Gentner, D., & Markman, A. B. (1997) · American Psychologist · doi:10.1037//0003-066x.52.1.45 · Cited by 1,816
(2006) · Designerly Ways of Knowing · doi:10.1007/1-84628-301-9_1 · Cited by 1,815
Proceedings of the 29th Annual Conference of the Cognitive Science Society
Rohde, H., Kehler, A., & Elman, J. L. (2007) · Cognitive Science · Cited by 1,788
Cross, N. (1982) · Design Studies · doi:10.1016/0142-694x(82)90040-0 · Cited by 1,499
Applied imagination : principles and procedures of creative problem-solving /
Osborn, A. F. (1963) · Cited by 1,445
(2007) · International Linear Collider · Cited by 1,382
Kolodner, J. (1993) · Case-Based Reasoning · doi:10.1016/b978-1-55860-237-3.50005-4 · Cited by 1,315
Jansson, D. G., & Smith, S. M. (1991) · Design Studies · doi:10.1016/0142-694x(91)90003-f · Cited by 1,087
Reliability, Validity, and Factor Structure of the Creative Achievement Questionnaire
Carson, S. H., Peterson, J. B., & Higgins, D. M. (2005) · Creativity Research Journal · doi:10.1207/s15326934crj1701_4 · Cited by 1,023
Learning and transfer: A general role for analogical encoding.
Gentner, D., Loewenstein, J., & Thompson, L. (2003) · Journal of Educational Psychology · doi:10.1037/0022-0663.95.2.393 · Cited by 865
Bono, E. D. (1985) · Cited by 780
Amazon Mechanical Turk: Gold Mine or Coal Mine?
Fort, K., Adda, G., & Cohen, K. B. (2011) · Computational Linguistics · doi:10.1162/coli_a_00057 · Cited by 761
Expertise in design: an overview
Cross, N. (2004) · Design Studies · doi:10.1016/j.destud.2004.06.002 · Cited by 644
The structure of design problem spaces
Goel, V (1992) · Cognitive Science · doi:10.1016/0364-0213(92)90038-v · Cited by 642
Serious play: how the world's best companies simulate to innovate
(2000) · Choice Reviews Online · doi:10.5860/choice.37-5189 · Cited by 606
Mele, A. R. (2003) · doi:10.1093/019515617x.001.0001 · Cited by 545
Summary knowledge of results for skill acquisition: Support for the guidance hypothesis.
Schmidt, R. A., Young, D. E., Swinnen, S., & Shapiro, D. C. (1989) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.15.2.352 · Cited by 407 · cited 1×
“However, delay helps learners re.ect: readily available, immediate feedback can be a crutch that discourages building a deep understanding [Anderson and Schooler 1990; Schmidt et al. 1989].”
BackgroundSimulated annealing: a proof of convergence
Granville, V., Krivanek, M., & Rasson, J.-P. (1994) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.295910 · Cited by 368
The Sciences of the Artificial - 3rd Edition
Simon, H. A. (1981) · MIT Press Books · Cited by 343
(2000) · Medical Entomology and Zoology · Cited by 339
Self-esteem and the cognitive accessibility of strengths and weaknesses after failure.
Dodgson, P. G., & Wood, J. V. (1998) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.75.1.178 · Cited by 313
Reflective physical prototyping through integrated design, test, and analysis
Hartmann, B., Klemmer, S. R., Bernstein, M., Abdulla, L., Burr, B., Robinson-Mosher, A., & Gee, J. (2006) · Proceedings of the 19th annual ACM symposium on User interface software and technology · doi:10.1145/1166253.1166300 · Cited by 302 · cited 2×
“Parallel Prototyping Leads to Better Design Results, More Divergence, and Increased Self-Ef.cacy STEVEN P. DOW, ALANA GLASSCO, JONATHAN KASS, MELISSA SCHWARZ, DANIEL L. SCHWARTZ, and SCOTT R. KLEMMER Stanford University Iteration can help people improve ideas.”
BackgroundSelf-citationGetting inspired!: understanding how and why examples are used in creative design practice
Herring, S. R., Chang, C.-C., Krantzler, J., & Bailey, B. P. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518717 · Cited by 263
Thompson, L., Gentner, D., & Loewenstein, J. (2000) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.2000.2887 · Cited by 243
Torrance Tests of Creative Thinking
Dumont, R., Willis, J. O., Veizel, K., & Zibulsky, J. (2014) · Encyclopedia of Special Education · doi:10.1002/9781118660584.ese2410 · Cited by 235
Getting the right design and the design right
Tohidi, M., Buxton, W., Baecker, R., & Sellen, A. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124960 · Cited by 211 · cited 2×
“…combat this, creating multiple alternatives in parallel may encourage people to more effectively discover unseen constraints and opportunities [Cross 2006], enumerate more diverse solutions [Buxton 2007], and obtain more authentic and diverse feedback from potential users [Tohidi et al. 2006].”
BackgroundMultiple Causes of Difficulty in Insight: The Case of the Nine-Dot Problem.
Kershaw, T. C., & Ohlsson, S. (2004) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.30.1.3 · Cited by 201
Designing with interactive example galleries
Lee, B., Srivastava, S., Kumar, R., Brafman, R., & Klemmer, S. R. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753667 · Cited by 195
Self-citationVisualization Criticism - The Missing Link Between Information Visualization and Art
Kosara, R. (2007) · 2007 11th International Conference Information Visualization (IV '07) · doi:10.1109/iv.2007.130 · Cited by 174
Tyler, M. E., & Ledford, J. (2006) · Cited by 173
The efficacy of prototyping under time constraints
Dow, S. P., Heddleston, K., & Klemmer, S. R. (2009) · Proceedings of the seventh ACM conference on Creativity and cognition · doi:10.1145/1640233.1640260 · Cited by 166 · cited 4×
“Yet, its primary virtue incremental, situated feedback can also blind designers to other alternatives, steering them to local, rather than global, optima [Buxton 2007; Dow et al. 2009].”
BackgroundInfluentialSelf-citationProblem-solving Strategies and Expertise in Engineering Design
Ball, L. J., St.B.T. Evans, J., Dennis, I., & Ormerod, T. C. (1997) · Thinking & Reasoning · doi:10.1080/135467897394284 · Cited by 158
Structured and opportunistic processing in design: a critical discussion
Ball, L. J., & Ormerod, T. C. (1995) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1995.1038 · Cited by 117
Graphic thinking for architects and designers
Fowles, B. (1982) · Design Studies · doi:10.1016/0142-694x(82)90014-x · Cited by 94
Web Metrics: Proven Methods for Measuring Web Site Success
(2003) · Performance Measurement and Metrics · doi:10.1108/14678040310507905 · Cited by 79
Web Metrics: Proven Methods for Measuring Web Site Success
Sterne, J. (2002) · Cited by 72
The Disruptive Potential of Immediate Feedback
Schooler, L. J., & Anderson, J. R. (2018) · KiltHub Repository · doi:10.1184/r1/6618494 · Cited by 64
Comparison and the development of knowledge
Boroditsky, L. (2007) · Cognition · doi:10.1016/j.cognition.2002.08.001 · Cited by 50
Improving system usability through parallel design
Nielsen, J., & Faber, J. (1996) · Computer · doi:10.1109/2.485844 · Cited by 46
Improving Self-Efficacy in Problem Solving: Learning from Errors and Feedback
Hall, T. S. (2008) · NC Digital Online Collection of Knowledge and Scholarship (The University of North Carolina at Greensboro) · Cited by 13
Learning abstract principles through principle-case comparison
Colhoun, J., & Dedre Gentner, C. (2008) · Cited by 12
Mele, A. R. (2005) · Philosophical Studies · doi:10.1007/s11098-004-4903-0 · Cited by 5
Motivation and Agency: Replies
Mele, A. R. (2005) · Philosophical Studies · doi:10.1007/s11098-004-4907-9 · Cited by 1 · cited 1×
“High self-ef.cacy improves one s ability to learn [Dweck 2007], perform tasks [Bandura 1997], exert agency and persist [Mele 2005], and .nd enjoyment in challenges [Csikszentmihalyi 1991].”
Background
Research works printed but not linked: 5
- 10.1037/0022-3514.75.1.178 · doi:10.1037/0022-3514.75.1.178 · Paper page
- 10.1037/0003-066x.52.1.45 · doi:10.1037/0003-066x.52.1.45 · Paper page
- 10.1207/s15516709cog1603_3 · doi:10.1207/s15516709cog1603_3 · Paper page
- Kelley , T. 2002. The Art of Innovation . Broadway Business Books . Kelley, T. 2002. The Art of Innovation. Broadway Business Books.
- Smith , S. , Kohn , N. , and Shah , J . 2008. What you see is what you get: Effects of provocative stimuli on creative invention . In Proceedings of the NSF International Workshop on Studying Design Creativity. Smith, S., Kohn, N., and Shah, J. 2008. What you see is what you get: Effects of provocative stimuli on creative invention. In Proceedings of the NSF International Workshop on Studying Design Creativity.
Cited by
Design Study Methodology: Reflections from the Trenches and the Stacks
Sedlmair, M., Meyer, M., & Munzner, T. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.213 · Cited by 826
“We conducted an extensive literature review in the fields of human computer interaction (HCI) [7, 8, 9, 12, 16, 19, 20, 21, 22, 25, 26, 27, 28, 29, 30, 31, 38, 47, 57, 63, 64, 65, 83] and social science [6, 14, 18, 24, 32, 62, 81, 87, 93] in hopes of finding methodologies that we could apply directly…”
BackgroundMethodIncremental and Radical Innovation: Design Research vs. Technology and Meaning Change
Norman, D. A., & Verganti, R. (2014) · Design Issues · doi:10.1162/desi_a_00250 · Cited by 773
Peer and self assessment in massive online classes
Kulkarni, C., Wei, K. P., Le, H., Chia, D., Papadopoulos, K., Cheng, J., Koller, D., & Klemmer, S. R. (2013) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2505057 · Cited by 390
“Inspired by (Dow et al. 2010), we call the result “fortune-cookie feedback” for its brevity and general applicability.”
MethodSelf-citationMummah, S. A., Robinson, T. N., King, A. C., Gardner, C. D., & Sutton, S. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5927 · Cited by 388
“It has been suggested that design thinking may increase the self-efficacy of those who practice the process [29] and frees even the most novice designers to generate more creative ideas and solutions.”
BackgroundDesign prototyping methods: state of the art in strategies, techniques, and guidelines
Camburn, B., Viswanathan, V., Linsey, J., Anderson, D., Jensen, D., Crawford, R., Otto, K., & Wood, K. (2017) · Design Science · doi:10.1017/dsj.2017.10 · Cited by 354
Metadata
- Authors
- Dow, S. P., Glassco, A., Kass, J., Schwarz, M., Schwartz, D. L., & Klemmer, S. R.
- Affiliations
- Stanford University (US)
- Venue
- ACM Transactions on Computer-Human Interaction
- Published
- 2010-12
- Volume
- 17
- Issue
- 4
- Pages
- 1-24
- ISSN
- 1073-0516 (ISSN-L)
- 1557-7325
- OpenAlex
- W2053859636
- Cited by
- 367
- References
- 74
- Citations by year
- 2026: 25 · 2025: 34 · 2024: 28 · 2023: 21 · 2022: 21 · 2021: 16 · 2020: 31 · 2019: 25 · 2018: 26 · 2017: 29 · 2016: 16 · 2015: 25 · 2014: 28 · 2013: 17 · 2012: 13
- FWCI
- 24.96
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Computer-Human Interaction (publishedVersion)
- Topics
- Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- design iteration, design fixation, self-efficacy, design exploration, feedback timing, novice designers, click-through data, design education, design diversity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.