The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Crowd science: The organization of scientific research in open collaborative projects
Open accessAbstract
A growing amount of scientific research is done in an open collaborative fashion, in projects sometimes referred to as “crowd science”, “citizen science”, or “networked science”. This paper seeks to gain a more systematic understanding of crowd science and to provide scholars with a conceptual framework and an agenda for future research. First, we briefly present three case examples that span different fields of science and illustrate the heterogeneity concerning what crowd science projects do and how they are organized. Second, we identify two fundamental elements that characterize crowd science projects – open participation and open sharing of intermediate inputs – and distinguish crowd science from other knowledge production regimes such as innovation contests or traditional “Mertonian” science. Third, we explore potential knowledge-related and motivational benefits that crowd science offers over alternative organizational modes, and potential challenges it is likely to face. Drawing on prior research on the organization of problem solving, we also consider for what kinds of tasks particular benefits or challenges are likely to be most pronounced. We conclude by outlining an agenda for future research and by discussing implications for funding agencies and policy makers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Structure of Scientific Revolutions
Kuhn, T. S., & Schlegel, R. (1963) · Physics Today · doi:10.1063/1.3050879 · Cited by 35,538
The Structure of Scientific Revolutions: 50th Anniversary Edition
Kuhn, T. S., & Hacking, I. (2012) · doi:10.7208/chicago/9780226458144.001.0001 · Cited by 21,170
Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions
Ryan, R. M., & Deci, E. L. (2000) · Contemporary Educational Psychology · doi:10.1006/ceps.1999.1020 · Cited by 18,666
(2003) · Cited by 9,368
The Architecture of Complexity
Simon, H. A. (1991) · Facets of Systems Science · doi:10.1007/978-1-4899-0718-9_31 · Cited by 4,433
The Sociology of Science: Theoretical and Empirical Investigations
Kaplan, N., Merton, R. K., & Storer, N. W. (1974) · Technology and Culture · doi:10.2307/3102264 · Cited by 4,005
The Increasing Dominance of Teams in Production of Knowledge
Wuchty, S., Jones, B. F., & Uzzi, B. (2007) · Science · doi:10.1126/science.1136099 · Cited by 3,615
Job Matching and the Theory of Turnover
Jovanovic, B. (1979) · Journal of Political Economy · doi:10.1086/260808 · Cited by 3,092
Cohen, W., Nelson, R., & Walsh, J. (2000) · National Bureau of Economic Research · doi:10.3386/w7552 · Cited by 3,025
Laboratory Life: The Social Construction of Scientific Facts
Latour, B., & Woolgar, S. (1986) · Oxford University Research Archive (ORA) (University of Oxford) · Cited by 2,980
Technology Brokering and Innovation in a Product Development Firm
Hargadon, A., & Sutton, R. I. (1997) · Administrative Science Quarterly · doi:10.2307/2393655 · Cited by 2,888
Laboratory life: The social construction of scientific facts
Austin, J. (1982) · Social Science & Medicine · doi:10.1016/0277-9536(82)90228-3 · Cited by 2,883
Recombinant Uncertainty in Technological Search
Fleming, L. (2001) · Management Science · doi:10.1287/mnsc.47.1.117.10671 · Cited by 2,873
Open Innovation: The New Imperative For Creating and Profiting From Technology
Mazzocchi, S. (2004) · Innovation · doi:10.5172/impp.2004.6.3.474 · Cited by 2,652
The structure of ill structured problems
Simon, H. A. (1973) · Artificial Intelligence · doi:10.1016/0004-3702(73)90011-8 · Cited by 2,600
Can Patents Deter Innovation? The Anticommons in Biomedical Research
Heller, M. A., & Eisenberg, R. S. (1998) · Science · doi:10.1126/science.280.5364.698 · Cited by 2,572
Toward a new economics of science
Partha, D., & David, P. A. (1994) · Research Policy · doi:10.1016/0048-7333(94)01002-1 · Cited by 2,448
Dahlander, L., & Gann, D. M. (2010) · Research Policy · doi:10.1016/j.respol.2010.01.013 · Cited by 2,435
Lorenzen, M. (2009) · Creativity, Innovation and the Cultural Economy · doi:10.4324/9780203880012-4a · Cited by 2,435
Open source software and the “private-collective” innovation model: Issues for organization science
Hippel, E. V., & Krogh, G. V. (2003) · Organization Science · doi:10.1287/orsc.14.2.209.14992 · Cited by 2,145
Crowdsourcing as a Model for Problem Solving: An Introduction and Cases
Brabham, D. C. (2008) · Convergence: The International Journal of Research into New Media Technologies · doi:10.1177/1354856507084420 · Cited by 1,943
Collaboration and Creativity: The Small World Problem
Uzzi, B., & Spiro, J. (2005) · American Journal of Sociology · doi:10.1086/432782 · Cited by 1,885
(1973) · Cited by 1,638
Lakhani, K. R., & Wolf, R. G. (2003) · doi:10.2139/ssrn.443040 · Cited by 1,531
Lakhani, K. R., & Wolf, R. G. (2005) · Perspectives on Free and Open Source Software · doi:10.7551/mitpress/5326.003.0005 · Cited by 1,530
THE SOCIOLOGY OF SCIENCE:: Theoretical and Empirical Investigations
Merton, R. Κ., & Storer, N. W. (2017) · Cited by 1,462
Predicting protein structures with a multiplayer online game
Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., Popović, Z., & players, F. (2010) · Nature · doi:10.1038/nature09304 · Cited by 1,368
A knowledge-based theory of the firm—The problem-solving perspective
Nickerson, J. A., & Zenger, T. R. (2004) · Organization Science · doi:10.1287/orsc.1040.0093 · Cited by 1,342
Hertel, G., Niedner, S., & Herrmann, S. (2003) · Research Policy · doi:10.1016/s0048-7333(03)00047-7 · Cited by 1,262
Crowdsourcing As a Solution to Distant Search
Afuah, A., & Tucci, C. L. (2012) · Academy of Management Review · doi:10.5465/amr.2010.0146 · Cited by 1,234
Marginality and Problem-Solving Effectiveness in Broadcast Search
Jeppesen, L. B., & Lakhani, K. R. (2010) · Organization Science · doi:10.1287/orsc.1090.0491 · Cited by 1,178
The Burden of Knowledge and the “Death of the Renaissance Man”: Is Innovation Getting Harder?
JONES, B. F. (2009) · Review of Economic Studies · doi:10.1111/j.1467-937x.2008.00531.x · Cited by 1,058
Community, joining, and specialization in open source software innovation: a case study
von Krogh, G., Spaeth, S., & Lakhani, K. R. (2003) · Research Policy · doi:10.1016/s0048-7333(03)00050-7 · Cited by 1,025
Working for Free? Motivations for Participating in Open-Source Projects
(2002) · International Journal of Electronic Commerce · doi:10.1080/10864415.2002.11044241 · Cited by 955
Stephan, P. (2012) · Harvard University Press eBooks · doi:10.4159/harvard.9780674062757 · Cited by 934
The Emergence of Governance in an Open Source Community
O'Mahony, S., & Ferraro, F. (2007) · Academy of Management Journal · doi:10.5465/amj.2007.27169153 · Cited by 909
Stephan, P. E. (2010) · Handbook of the Economics of Innovation · doi:10.1016/s0169-7218(10)01005-1 · Cited by 906
Motivation, Governance, and the Viability of Hybrid Forms in Open Source Software Development
Shah, S. K. (2006) · Management Science · doi:10.1287/mnsc.1060.0553 · Cited by 894
Incentives and Problem Uncertainty in Innovation Contests: An Empirical Analysis
Boudreau, K. J., Lacetera, N., & Lakhani, K. R. (2011) · Management Science · doi:10.1287/mnsc.1110.1322 · Cited by 830
Lone Inventors as Sources of Breakthroughs: Myth or Reality?
Singh, J., & Fleming, L. (2010) · Management Science · doi:10.1287/mnsc.1090.1072 · Cited by 758
Creativity: Understanding Innovation in Problem Solving, Science, Invention, and the Arts
Weisberg, R. W. (2006) · Cited by 750
The market economy, and the scientific commons
Nelson, R. R. (2004) · Research Policy · doi:10.1016/j.respol.2003.09.008 · Cited by 747
Galaxy Zoo 1: data release of morphological classifications for nearly 900 000 galaxies★
Lintott, C., Schawinski, K., Bamford, S., Slosar, A., Land, K., Thomas, D., Edmondson, E., Masters, K., Nichol, R. C., Raddick, M. J., et al. (2011) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2010.17432.x · Cited by 738
From Conservation to Crowdsourcing: A Typology of Citizen Science
Wiggins, A., & Crowston, K. (2011) · 2011 44th Hawaii International Conference on System Sciences · doi:10.1109/hicss.2011.207 · Cited by 700
Carrots and Rainbows: Motivation and Social Practice in Open Source Software Development1
von Krogh, G., Haefliger, S., Spaeth, S., & Wallin, M. W. (2012) · MIS Quarterly · doi:10.2307/41703471 · Cited by 698
Why Open Source software can succeed
Bonaccorsi, A., & Rossi, C. (2003) · Research Policy · doi:10.1016/s0048-7333(03)00051-9 · Cited by 678
Reinventing discovery: the new era of networked science
(2012) · Choice Reviews Online · doi:10.5860/choice.49-4420 · Cited by 645
Algorithm discovery by protein folding game players
Khatib, F., Cooper, S., Tyka, M. D., Xu, K., Makedon, I., Popović, Z., Baker, D., & Players, F. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1115898108 · Cited by 582
Galaxy Zoo Green Peas: discovery of a class of compact extremely star-forming galaxies
Cardamone, C., Schawinski, K., Sarzi, M., Bamford, S. P., Bennert, N., Urry, C. M., Lintott, C., Keel, W. C., Parejko, J., Nichol, R. C., et al. (2009) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2009.15383.x · Cited by 569
Crystal structure of a monomeric retroviral protease solved by protein folding game players
Foldit Contenders Group, Khatib, F., Foldit Void Crushers Group, DiMaio, F., Cooper, S., Kazmierczyk, M., Gilski, M., Krzywda, S., Zabranska, H., Pichova, I., et al. (2011) · Nature Structural & Molecular Biology · doi:10.1038/nsmb.2119 · Cited by 540
Do formal intellectual property rights hinder the free flow of scientific knowledge?
Murray, F., & Stern, S. (2007) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2006.05.017 · Cited by 514
Baldwin, C. Y., & Clark, K. B. (2006) · Management Science · doi:10.1287/mnsc.1060.0546 · Cited by 500
Replication in Empirical Economics: The Journal of Money, Credit and Banking Project
Dewald, W. G., Thursby, J. G., & Anderson, R. G. (2016) · American Economic Review · Cited by 477
Galaxy Zoo: Exploring the Motivations of Citizen Science Volunteers
Raddick, M. J., Bracey, G., Gay, P. L., Lintott, C. J., Murray, P., Schawinski, K., Szalay, A. S., & Vandenberg, J. (2010) · Astronomy Education Review · doi:10.3847/aer2009036 · Cited by 473
The impact of ideology on effectiveness in open source software development teams
Stewart, K., & Gosain, S. (2006) · doi:10.5555/2017307.2017313 · Cited by 471
Coordination costs and project outcomes in multi-university collaborations
Cummings, J. N., & Kiesler, S. (2007) · Research Policy · doi:10.1016/j.respol.2007.09.001 · Cited by 410
When Authorship Fails: A Proposal to Make Contributors Accountable
Rennie, D. (1997) · JAMA · doi:10.1001/jama.1997.03550070071041 · Cited by 396
The Economics of Technology Sharing: Open Source and Beyond
Lerner, J., & Tirole, J. (2005) · Journal of Economic Perspectives · doi:10.1257/0895330054048678 · Cited by 395
Science and the diffusion of knowledge
Sorenson, O., & Fleming, L. (2004) · Research Policy · doi:10.1016/j.respol.2004.09.008 · Cited by 387
What Makes Them Tick? Employee Motives and Firm Innovation
Sauermann, H., & Cohen, W. M. (2010) · Management Science · doi:10.1287/mnsc.1100.1241 · Cited by 378
Self-citationConflicting Logics? A Multidimensional View of Industrial and Academic Science
Sauermann, H., & Stephan, P. (2013) · Organization Science · doi:10.1287/orsc.1120.0769 · Cited by 350
Self-citationCode Reuse in Open Source Software
Haefliger, S., von Krogh, G., & Spaeth, S. (2008) · Management Science · doi:10.1287/mnsc.1070.0748 · Cited by 333
The Scientific Life: A Moral History of a Late Modern Vocation
Labinger, J. A. (2009) · Physics Today · doi:10.1063/1.3099581 · Cited by 316
Open source software development—Just another case of collective invention?
Osterloh, M., & Rota, S. (2007) · Research Policy · doi:10.1016/j.respol.2006.10.004 · Cited by 281
Exploring the Foundations of Cumulative Innovation: Implications for Organization Science
Murray, F., & O'Mahony, S. (2007) · Organization Science · doi:10.1287/orsc.1070.0325 · Cited by 278
View from the Bench: Patents and Material Transfers
Walsh, J. P., Cho, C., & Cohen, W. M. (2005) · Science · doi:10.1126/science.1115813 · Cited by 228
Galaxy Zoo: Motivations of Citizen Scientists
Jordan Raddick, M., Bracey, G., Gay, P. L., Lintott, C. J., Cardamone, C., Murray, P., Schawinski, K., Szalay, A. S., & Vandenberg, J. (2013) · Astronomy Education Review · doi:10.3847/aer2011021 · Cited by 217
Progressing to the Center: Coordinating Project Work
Dahlander, L., & O'Mahony, S. (2011) · Organization Science · doi:10.1287/orsc.1100.0571 · Cited by 211
Zuckerman, H. (1988) · Cited by 205
Dusting for science: motivation and participation of digital citizen science volunteers
Nov, O., Arazy, O., & Anderson, D. (2011) · iConference · doi:10.1145/1940761.1940771 · Cited by 167
Perspectives on Free and Open Source Software
Guadamuz Gonzalez, A. (2006) · SCRIPT-ed · doi:10.2966/scrip.030406.485 · Cited by 166
Strong regularities in online peer production
Wilkinson, D. M. (2008) · Proceedings of the 9th ACM conference on Electronic commerce · doi:10.1145/1386790.1386837 · Cited by 160
Dynamic Bayesian Combination of Multiple Imperfect Classifiers
Simpson, E., Roberts, S., Psorakis, I., & Smith, A. M. (2013) · Studies in computational intelligence · doi:10.1007/978-3-642-36406-8_1 · Cited by 130
Harnessing Collective Intelligence to Address Global Climate Change
Malone, T. W., & Klein, M. (2007) · Innovations Technology Governance Globalization · doi:10.1162/itgg.2007.2.3.15 · Cited by 124
The Economics of Technology Sharing: Open Source and Beyond
Lerner, J., & Tirole, J. (2004) · doi:10.3386/w10956 · Cited by 122
Haeussler, C., & Sauermann, H. (2012) · Research Policy · doi:10.1016/j.respol.2012.09.009 · Cited by 116
Self-citationThe Increasing Dominance of Teams in the Production of KnowledgePreprint
Wuchty, S., Jones, B. F., & Uzzi, B. (2007) · SSRN Electronic Journal · Cited by 114
Not all scientists pay to be scientists: PhDs’ preferences for publishing in industrial employment
Sauermann, H., & Roach, M. (2013) · Research Policy · doi:10.1016/j.respol.2013.07.006 · Cited by 113
Self-citationLegal Implications of Open-Source SoftwarePreprint
McGowan, D. (2000) · SSRN Electronic Journal · doi:10.2139/ssrn.243237 · Cited by 102
The periphery on stage: The intra-organizational dynamics in online communities of creation
Rullani, F., & Haefliger, S. (2013) · Research Policy · doi:10.1016/j.respol.2012.10.008 · Cited by 79
Geneticists push for global data-sharing
Hayden, E. C. (2013) · Nature · doi:10.1038/498017a · Cited by 75
Alexy, O. T., & Leitner, M. (2011) · European Management Review · doi:10.1111/j.1740-4762.2011.01017.x · Cited by 73
Hanging with the right crowd: Matching crowdsourcing need to crowd characteristics
Erickson, L. B., Petrick, I. J., & Trauth, E. M. (2012) · Journal of the Association for Information Systems · Cited by 70
Lone Inventors as Source of Breakthroughs: Myth or Reality?Preprint
Singh, J., & Fleming, L. (2008) · SSRN Electronic Journal · doi:10.2139/ssrn.1299064 · Cited by 66
Open Source Drug Discovery in Practice: A Case Study
Årdal, C., & Røttingen, J. (2012) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0001827 · Cited by 61
Prestopnik, N. R., & Crowston, K. (2011) · IEEE International Conference on e-Science Workshops · doi:10.1109/esciencew.2011.14 · Cited by 58
Deterministic methods to find primes
Tao, T., Croot, E. S., & Helfgott, H. A. (2011) · Mathematics of Computation · doi:10.1090/s0025-5718-2011-02542-1 · Cited by 41
Specific and General Information Sharing Among Academic Scientists
Haeussler, C., Jiang, L., Thursby, J. G., & Thursby, M. (2009) · National Bureau of Economic Research · doi:10.3386/w15315 · Cited by 36
Open or Closed Innovation? Problem Solving and Governance ChoicePreprint
Felin, T., & Zenger, T. R. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2156951 · Cited by 13
Raasch, C., & von Hippel, E. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2167948 · Cited by 11
10 Open, But How Much? Growth,Conflict, and Institutional Evolution in Open‐Source Communities1
Mateos-Garcia, J., & Steinmueller, W. E. (2008) · Community, Economic Creativity, and Organization · doi:10.1093/acprof:oso/9780199545490.003.0011 · Cited by 8
Is Collaboration Creative or Costly? Exploring Tradeoffs in the Organization of Knowledge WorkPreprint
Bikard, M. A., & Murray, F. (2011) · SSRN Electronic Journal · doi:10.2139/ssrn.1953324 · Cited by 6
The Economics of Scientific Misconduct
Lacetera, N., & Zirulia, L. (2011) · Journal of Law, Economics, and Organization · doi:10.1093/jleo/ewp031 · Cited by 5
Cited by
Digital innovation and transformation: An institutional perspective
Hinings, B., Gegenhuber, T., & Greenwood, R. (2018) · Information and Organization · doi:10.1016/j.infoandorg.2018.02.004 · Cited by 1,537
The specificity of the scientific field and the social conditions of the progress of reason
Bourdieu, P. (1975) · Social Science Information · doi:10.1177/053901847501400602 · Cited by 1,483
Bogers, M., Zobel, A.-K., Afuah, A., Almirall, E., Brunswicker, S., Dahlander, L., Frederiksen, L., Gawer, A., Gruber, M., Haefliger, S., et al. (2017) · Industry and Innovation · doi:10.1080/13662716.2016.1240068 · Cited by 1,001
Incentives and Problem Uncertainty in Innovation Contests: An Empirical Analysis
Boudreau, K. J., Lacetera, N., & Lakhani, K. R. (2011) · Management Science · doi:10.1287/mnsc.1110.1322 · Cited by 830
What Is Citizen Science? – A Scientometric Meta-Analysis
Kullenberg, C., & Kasperowski, D. (2016) · PLOS ONE · doi:10.1371/journal.pone.0147152 · Cited by 716
Metadata
- Authors
- Franzoni, C., & Sauermann, H.
- Venue
- Research Policy
- Publisher
- Elsevier BV
- Published
- 2014-02
- Volume
- 43
- Issue
- 1
- Pages
- 1-20
- ISSN
- 0048-7333 (ISSN-L)
- OpenAlex
- W3124163132
- Cited by
- 505
- References
- 93
- FWCI
- 57.06
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.