The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Crowd science user contribution patterns and their implications
Open accessAbstract
Significance: Involving the public in research may provide considerable benefits for the progress of science. However, the sustainability of “crowd science” approaches depends on the degree to which members of the public are interested and provide continued labor inputs. We describe and compare contribution patterns in multiple projects using a range of measures. We show that effort contributions can be significant in magnitude and speed, but we also identify several challenges. In addition, we explore some of the underlying dynamics and mechanisms. As such, we provide quantitative evidence that is useful for scientists who consider adopting crowd science approaches and for scholars studying crowd-based knowledge production. Our results also inform current policy discussions regarding the organization of scientific research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Architecture of Complexity
Simon, H. A. (1991) · Facets of Systems Science · doi:10.1007/978-1-4899-0718-9_31 · Cited by 4,433
The biodiversity of species and their rates of extinction, distribution, and protection
Pimm, S. L., Jenkins, C. N., Abell, R., Brooks, T. M., Gittleman, J. L., Joppa, L. N., Raven, P. H., Roberts, C. M., & Sexton, J. O. (2014) · Science · doi:10.1126/science.1246752 · Cited by 3,542 · cited 1×
“Fifth, crowd science projects can involve contributors across time and geographic space, allowing them to increase coverage that is particularly important for observational studies (14).”
Work group diversity and group performance: an integrative model and research agenda.
van Knippenberg, D., De Dreu, C. K. W., & Homan, A. C. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.6.1008 · Cited by 2,911
Predicting protein structures with a multiplayer online game
Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., Popović, Z., & players, F. (2010) · Nature · doi:10.1038/nature09304 · Cited by 1,368
Next Steps for Citizen Science
Bonney, R., Shirk, J. L., Phillips, T. B., Wiggins, A., Ballard, H. L., Miller-Rushing, A. J., & Parrish, J. K. (2014) · Science · doi:10.1126/science.1251554 · Cited by 1,253
Marginality and Problem-Solving Effectiveness in Broadcast Search
Jeppesen, L. B., & Lakhani, K. R. (2010) · Organization Science · doi:10.1287/orsc.1090.0491 · Cited by 1,178
The Estimation of the Lorenz Curve and Gini Index
Gastwirth, J. L. (1972) · The Review of Economics and Statistics · doi:10.2307/1937992 · Cited by 1,132
The Architecture of Complexity
(2007) · IEEE Control Systems · doi:10.1109/mcs.2007.384127 · Cited by 805
From Conservation to Crowdsourcing: A Typology of Citizen Science
Wiggins, A., & Crowston, K. (2011) · 2011 44th Hawaii International Conference on System Sciences · doi:10.1109/hicss.2011.207 · Cited by 700
Soylent: a word processor with a crowd inside
Bernstein, M. S., Little, G., Miller, R. C., Hartmann, B., Ackerman, M. S., Karger, D. R., Crowell, D., & Panovich, K. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866078 · Cited by 686 · cited 2×
“Many of these approaches decompose the complex task into smaller microtasks, distribute these microtasks to a large number of contributors, and then use sophisticated mechanisms to integrate the resulting contributions (28, 40).”
Reinventing discovery: the new era of networked science
(2012) · Choice Reviews Online · doi:10.5860/choice.49-4420 · Cited by 645
Galaxy Zoo Green Peas: discovery of a class of compact extremely star-forming galaxies
Cardamone, C., Schawinski, K., Sarzi, M., Bamford, S. P., Bennert, N., Urry, C. M., Lintott, C., Keel, W. C., Parejko, J., Nichol, R. C., et al. (2009) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2009.15383.x · Cited by 569
Crystal structure of a monomeric retroviral protease solved by protein folding game players
Foldit Contenders Group, Khatib, F., Foldit Void Crushers Group, DiMaio, F., Cooper, S., Kazmierczyk, M., Gilski, M., Krzywda, S., Zabranska, H., Pichova, I., et al. (2011) · Nature Structural & Molecular Biology · doi:10.1038/nsmb.2119 · Cited by 540
Crowd science: The organization of scientific research in open collaborative projects
Franzoni, C., & Sauermann, H. (2014) · Research Policy · doi:10.1016/j.respol.2013.07.005 · Cited by 505
Self-citationBaldwin, C. Y., & Clark, K. B. (2006) · Management Science · doi:10.1287/mnsc.1060.0546 · Cited by 500
Space–time wiring specificity supports direction selectivity in the retina
EyeWirers, T., Kim, J. S., Greene, M. J., Zlateski, A., Lee, K., Richardson, M., Turaga, S. C., Purcaro, M., Balkam, M., Robinson, A., et al. (2014) · Nature · doi:10.1038/nature13240 · Cited by 450
CrowdForge: crowdsourcing complex work
Kittur, A., Smus, B., & Kraut, R. (2011) · CHI '11 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1979742.1979902 · Cited by 434
Shepherding the crowd yields better work
Dow, S., Kulkarni, A., Klemmer, S., & Hartmann, B. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145355 · Cited by 334 · cited 1×
“For example, researchers have examined how the crowd can get involved in programming, decision making, or in the writing of nonscientific articles (28, 38, 40, 41).”
Key issues and new approaches for evaluating citizen‐science learning outcomes
Jordan, R., Ballard, H. L., & Phillips, T. (2012) · Frontiers in Ecology and the Environment · doi:10.1890/110280 · Cited by 333
CrowdForge: crowdsourcing complex work
Kittur, A., Smus, B., Khamkar, S., & Kraut, R. E. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047202 · Cited by 309
RNA design rules from a massive open laboratory
Lee, J., Kladwang, W., Lee, M., Cantu, D., Azizyan, M., Kim, H., Limpaecher, A., Gaikwad, S., Yoon, S., Treuille, A., et al. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1313039111 · Cited by 301
Wikipedians are born, not made: a study of power editors on Wikipedia
Panciera, K., Halfaker, A., & Terveen, L. (2009) · Proceedings of the 2009 ACM International Conference on Supporting Group Work · doi:10.1145/1531674.1531682 · Cited by 276
Phylo: A Citizen Science Approach for Improving Multiple Sequence Alignment
Kawrykow, A., Roumanis, G., Kam, A., Kwak, D., Leung, C., Wu, C., Zarour, E., players, P., Sarmenta, L., Blanchette, M., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0031362 · Cited by 234
Galaxy Zoo: Motivations of Citizen Scientists
Jordan Raddick, M., Bracey, G., Gay, P. L., Lintott, C. J., Cardamone, C., Murray, P., Schawinski, K., Szalay, A. S., & Vandenberg, J. (2013) · Astronomy Education Review · doi:10.3847/aer2011021 · Cited by 217
Authorship and contribution disclosures
Sauermann, H., & Haeussler, C. (2017) · Science Advances · doi:10.1126/sciadv.1700404 · Cited by 188
Self-citationOn the Inequality of Contributions to Wikipedia
Ortega, F., Gonzalez-Barahona, J. M., & Robles, G. (2008) · Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008) · doi:10.1109/hicss.2008.333 · Cited by 171
Strong regularities in online peer production
Wilkinson, D. M. (2008) · Proceedings of the 9th ACM conference on Electronic commerce · doi:10.1145/1386790.1386837 · Cited by 160
Dynamic Bayesian Combination of Multiple Imperfect Classifiers
Simpson, E., Roberts, S., Psorakis, I., & Smith, A. M. (2013) · Studies in computational intelligence · doi:10.1007/978-3-642-36406-8_1 · Cited by 130
Harnessing Collective Intelligence to Address Global Climate Change
Malone, T. W., & Klein, M. (2007) · Innovations Technology Governance Globalization · doi:10.1162/itgg.2007.2.3.15 · Cited by 124
Galaxy Zoo: Morphological Classification and Citizen SciencePreprint
Fortson, L., Masters, K. L., Nichol, R. C., Borne, K. D., Edmondson, E. M., Lintott, C. J., Raddick, J., Schawinski, K., & Wallin, J. F. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1104.5513 · Cited by 71
Jackson, C. B., Østerlund, C. S., Mugar, G., Hassman, K., & Crowston, K. (2015) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2015.196 · Cited by 61
Studying the laws of software evolution in a long‐lived FLOSS project
Gonzalez-Barahona, J. M., Robles, G., Herraiz, I., & Ortega, F. (2013) · Journal of Software Evolution and Process · doi:10.1002/smr.1615 · Cited by 40
Open-Phylo: a customizable crowd-computing platform for multiple sequence alignment
Kwak, D., Kam, A., Becerra, D., Zhou, Q., Hops, A., Zarour, E., Kam, A., Sarmenta, L. F. G., Blanchette, M., & Waldispühl, J. (2013) · Genome biology · doi:10.1186/gb-2013-14-10-r116 · Cited by 30
Fortson, L., Masters, K. L., Nichol, R. C., Borne, K. D., Edmondson, E. M., Lintott, C. J., Raddick, J., Schawinski, K., & Wallin, J. F. (2012) · Chapman & Hall/CRC data mining and knowledge discovery series · doi:10.1201/b11822-16 · Cited by 27
Participation in crowd systems
Varshney, L. R. (2012) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2012.6483327 · Cited by 22
The nature and composition of the Linux kernel developer community: a dynamic analysis
Ghosh, R. A. (2003) · Research Publications (Maastricht University) · Cited by 19
Haeussler, C., & Sauermann, H. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2434327 · Cited by 10
Self-citationSatisfying heterogeneous user needs via innovation toolkits: the case of Apache Security Software
Franke, N., & Hippel, E. V. (2003) · Research Policy · doi:10.1016/s0048-7333(03)00049-0 · Cited by 2
Cited by
Can citizen science enhance public understanding of science?
Bonney, R., Phillips, T. B., Ballard, H. L., & Enck, J. W. (2016) · Public Understanding of Science · doi:10.1177/0963662515607406 · Cited by 994
Swanson, A., Kosmala, M., Lintott, C., Simpson, R., Smith, A., & Packer, C. (2015) · Scientific Data · doi:10.1038/sdata.2015.26 · Cited by 510
Citizen science: a new approach to advance ecology, education, and conservation
Kobori, H., Dickinson, J. L., Washitani, I., Sakurai, R., Amano, T., Komatsu, N., Kitamura, W., Takagawa, S., Koyama, K., Ogawara, T., et al. (2016) · Ecological Research · doi:10.1007/s11284-015-1314-y · Cited by 436
“Citizen Science”? Rethinking Science and Public Participation
Strasser, B. J., Baudry, J., Mahr, D., Sanchez, G., & Tancoigne, E. (2018) · Science & Technology Studies · doi:10.23987/sts.60425 · Cited by 425
The diversity and evolution of ecological and environmental citizen science
Pocock, M. J. O., Tweddle, J. C., Savage, J., Robinson, L. D., & Roy, H. E. (2017) · PLOS ONE · doi:10.1371/journal.pone.0172579 · Cited by 359
Metadata
- Authors
- Sauermann, H., & Franzoni, C.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2015-01-05
- Volume
- 112
- Issue
- 3
- Pages
- 679-684
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W1993502890
- Cited by
- 305
- References
- 38
- FWCI
- 48.75
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.