Borrowing from the Crowd: A Study of Recombination in Software Design Competitions
Abstract
One form of crowdsourcing is the competition, which poses an open call for competing solutions. Commercial systems such as TopCoder have begun to explore the application of competitions to software development, but have important limitations diminishing the potential benefits drawn from the crowd. In particular, they employ a model of independent work that ignores the opportunity for designs to arise from the ideas of multiple designers. In this paper, we examine the potential for software design competitions to incorporate recombination, in which competing designers are given the designs of others and encouraged to use them to revise their own designs. To explore this, we conducted two software design competitions in which participants were asked to produce both an initial and a revised design, drawing on lessons learned from the crowd. We found that, in both competitions, all participants borrowed ideas and most improved the quality of their designs. Our findings demonstrate the potential benefits of recombination in software design and suggest several ways in which software design competitions can be improved.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Low and top performers appear to benefit similarly from public submissions and peer comments.
“Importantly, low and top performers appear to benefit similarly from such an arrangement (LaToza et al., 2015).”
From Relative Feedback Increases Disparities in Effort and Performance in Crowdsourcing Contests (2022) · cites this paper as LaToza et al., 2015
The citation lists haven’t been fetched yet
References 31 · Cited by 23
- Authors
- LaToza, T. D., Chen, M., Jiang, L., Zhao, M., & Hoek, A. V. D.
- Affiliations
- University of California, Irvine (US)
- University of Amsterdam (NL)
- ORCID
- LaToza, Thomas D.
- Venue
- 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering
- Publisher
- IEEE
- Published
- 2015-05
- Pages
- 551-562
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W4249454228
- Cited by
- 23
- References
- 31
- Citations by year
- 2021: 1 · 2020: 9 · 2019: 3 · 2018: 3 · 2017: 2 · 2016: 1 · 2015: 4
- FWCI
- 2.38
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Open Source Software Innovations
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Keywords
- crowdsourcing, recombination, TopCoder, design quality, co-design, design changes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
