Algorithmic Management for Improving Collective Productivity in Crowdsourcing
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Crowdsourcing systems are complex not only because of the huge number of potential strategies for assigning workers to tasks, but also due to the dynamic characteristics associated with workers. Maximizing social welfare in such situations is known to be NP-hard. To address these fundamental challenges, we propose the surprise-minimization-value-maximization (SMVM) approach. By analysing typical crowdsourcing system dynamics, we established a simple and novel worker desirability index(WDI) jointly considering the effect of each worker’s reputation, workload and motivation to work on collective productivity. Through evaluating workers’ WDI values, SMVM influences individual workers in real time about courses of action which can benefit the workers and lead to high collective productivity. Solutions can be produced in polynomial time and are proven to be asymptotically bounded by a theoretical optimal solution. High resolution simulations based on a real-world dataset demonstrate that SMVM significantly outperforms state-of-the-art approaches. A large-scale 3-year empirical study involving 1,144 participants in over 9,000 sessions shows that SMVM outperforms human task delegation decisions over 80% of the time under common workload conditions. The approach and results can help engineer highly scalable data-driven algorithmic management decision support systems for crowdsourcing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The sharing economy: Why people participate in collaborative consumption
Hamari, J., Sjöklint, M., & Ukkonen, A. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23552 · Cited by 3,439
Computational principles of movement neuroscience
Wolpert, D. M., & Ghahramani, Z. (2000) · Nature Neuroscience · doi:10.1038/81497 · Cited by 2,122
Working with Machines: The Impact of Algorithmic and Data-Driven Management on Human Workers
Lee, M. K., Kusbit, D., Metsky, E., & Dabbish, L. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702548 · Cited by 823
Crystal structure of a monomeric retroviral protease solved by protein folding game players
Foldit Contenders Group, Khatib, F., Foldit Void Crushers Group, DiMaio, F., Cooper, S., Kazmierczyk, M., Gilski, M., Krzywda, S., Zabranska, H., Pichova, I., et al. (2011) · Nature Structural & Molecular Biology · doi:10.1038/nsmb.2119 · Cited by 540 · cited 2×
“By analysing typical crowdsourcing system dynamics, we establish a simple and novel worker desirability index (WDI) (equation (10)) which uses information on workers’ reputation28, workload and motivation to work29,30 to translate system-level objectives into actionable task delegation decisions for individual workers.”
MethodEvidence for surprise minimization over value maximization in choice behavior
Schwartenbeck, P., FitzGerald, T. H. B., Mathys, C., Dolan, R., Kronbichler, M., & Friston, K. (2015) · Scientific Reports · doi:10.1038/srep16575 · Cited by 96
Cited by
FedVision: An Online Visual Object Detection Platform Powered by Federated Learning
Liu, Y., Huang, A., Luo, Y., Huang, H., Liu, Y., Chen, Y., Feng, L., Chen, T., Yu, H., & Yang, Q. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i08.7021 · Cited by 310
Self-citationBuilding Ethics into Artificial Intelligence
Yu, H., Shen, Z., Miao, C., Leung, C., Lesser, V. R., & Yang, Q. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/779 · Cited by 210
Self-citationKinowska, H., & Sienkiewicz, Ł. J. (2023) · Information Technology & People · doi:10.1108/itp-02-2022-0079 · Cited by 147
Towards AI-Powered Personalization in MOOC Learning
Yu, H., Miao, C., Leung, C., & White, T. J. (2017) · npj Science of Learning · doi:10.1038/s41539-017-0016-3 · Cited by 131
Self-citationA sustainable incentive scheme for federated learning
Yu, H., Liu, Z., Liu, Y., Chen, T., Cong, M., Weng, X., Niyato, D., & Yang, Q. (2020) · IEEE Intelligent Systems · doi:10.1109/mis.2020.2987774 · Cited by 123
Self-citation
Metadata
- Authors
- Yu, H., Miao, C., Chen, Y., Fauvel, S., Li, X., & Lesser, V. R.
- Venue
- Scientific Reports
- Published
- 2017-10-02
- Volume
- 7
- Issue
- 1
- Pages
- 12541-12541
- ISSN
- 2045-2322 (ISSN-L)
- OpenAlex
- W2758374631
- Cited by
- 53
- References
- 5
- FWCI
- 8.55
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Algorithmic Management for Improving Collective Productivity in Crowdsourcing. https://app.cochat.ai/papers/10.1038/s41598-017-12757-x
- The paper
- Yu, H., Miao, C., Chen, Y., Fauvel, S., Li, X., & Lesser, V. R. (2017). Algorithmic Management for Improving Collective Productivity in Crowdsourcing. Scientific Reports. https://doi.org/10.1038/s41598-017-12757-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.