A technical survey on statistical modelling and design methods for crowdsourcing quality control
Open accessIntegrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most existing aggregation methods focus on weighting worker contributions without accounting for contextual information, though other methods introduce context-aware mechanisms.
“Most existing aggregation methods focus on weighting worker contributions without accounting for contextual information (Zhang et al., 2016;Zheng et al., 2017;Zhang, 2022)), though other methods introduce context-aware mechanisms (Jin et al., 2020;Zhang, 2022).”
From SkillAggregation: Reference-free LLM-Dependent Aggregation (2025) · cites this paper as Jin et al., 2020
Cites
Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions
Ryan, R. M., & Deci, E. L. (2000) · Contemporary Educational Psychology · doi:10.1006/ceps.1999.1020 · Cited by 18,666 · cited 1× · see citing works
“According to [103], motivations can be broadly characterized as being either extrinsic, which is the desire to gain monetary payoffs and avoid costs, or intrinsic, which is the desire to achieve fulfilment and enjoyment.”
BackgroundKittur, A., Nickerson, J. V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., & Horton, J. (2013) · Proceedings of the 2013 conference on Computer supported cooperative work · doi:10.1145/2441776.2441923 · Cited by 1,034 · see citing works
Incentives and Problem Uncertainty in Innovation Contests: An Empirical Analysis
Boudreau, K. J., Lacetera, N., & Lakhani, K. R. (2011) · Management Science · doi:10.1287/mnsc.1110.1322 · Cited by 830 · see citing works
Plackett, R. L. (1975) · Applied Statistics · doi:10.2307/2346567 · Cited by 731 · see citing works
Christy, K. R., & Fox, J. (2014) · Computers & Education · doi:10.1016/j.compedu.2014.05.005 · Cited by 276 · see citing works
Hakulinen, L., Auvinen, T., & Korhonen, A. (2015) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v10i1.4221 · Cited by 201 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
SkillAggregation: Reference-free LLM-Dependent Aggregation
Sun, G., Kagrecha, A., Manakul, P., Woodland, P., & Gales, M. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.756 · Cited by 3 · see citing works
- Authors
- Jin, Y., Carman, M., Zhu, Y., & Xiang, Y.
- Affiliations
- Monash University (AU)
- Politecnico di Milano (IT)
- Deakin University (AU)
- Corresponding author
- Jin, Yuan
- Venue
- Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 2020-10
- Volume
- 287
- Pages
- 103351
- ISSN
- 0004-3702 (ISSN-L)
- OpenAlex
- W3038629767
- Cited by
- 28
- References
- 6
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 3 · 2023: 7 · 2022: 4 · 2021: 5 · 2020: 1 · 2019: 2
- FWCI
- 3.73
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (vor, 1461 d)
- doi.org/10.15223/policy-017 (stm-asf)
- doi.org/10.15223/policy-037 (stm-asf)
- doi.org/10.15223/policy-012 (stm-asf)
- doi.org/10.15223/policy-029 (stm-asf)
- doi.org/10.15223/policy-004 (stm-asf)
- Locations
- Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Virtual Community of Pathological Anatomy (University of Castilla La Mancha) (submittedVersion, other-oa)
- Figshare (submittedVersion, cc-by)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Information Systems › Expert finding and Q&A systems
- Keywords
- response quality, response aggregation, employee behavior, gamification, payment mechanisms, performance guarantees
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review