Creativity on Paid Crowdsourcing Platforms
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Crowdsourcing platforms are increasingly being harnessed for creative work. The platforms' potential for creative work is clearly identified, but the workers' perspectives on such work have not been extensively documented. In this paper, we uncover what the workers have to say about creative work on paid crowdsourcing platforms. Through a quantitative and qualitative analysis of a questionnaire launched on two different crowdsourcing platforms, our results revealed clear differences between the workers on the platforms in both preferences and prior experience with creative work. We identify common pitfalls with creative work on crowdsourcing platforms, provide recommendations for requesters of creative work, and discuss the meaning of our findings within the broader scope of creativity-oriented research. To the best of our knowledge, we contribute the first extensive worker-oriented study of creative work on paid crowdsourcing platforms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
The standard definition of creativity
Runco, M. A., & Jaeger, G. J. (2012) · Creativity Research Journal · doi:10.1080/10400419.2012.650092 · Cited by 3,895 · cited 1×
“, in this case divergent thinking) has become a staple in the toolbox of many researchers [61].”
BackgroundGuilford, J. P. (1950) · American Psychologist · doi:10.1037/h0063487 · Cited by 3,588
Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.01.006 · Cited by 2,957 · cited 1×
“Prolific is a crowdsourcing platform targeted towards academic studies [56].”
BackgroundBeyond Big and Little: The Four C Model of Creativity
Kaufman, J. C., & Beghetto, R. A. (2009) · Review of General Psychology · doi:10.1037/a0013688 · Cited by 2,669
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Hauser, D. J., & Schwarz, N. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0578-z · Cited by 1,739
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Peer, E., Vosgerau, J., & Acquisti, A. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0434-y · Cited by 1,618 · cited 2×
“Similar qualification criteria are typically being used in academic studies to improve the chance of receiving high quality responses [57].”
BackgroundMcDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072
Cited by
Comparing attentional disengagement between prolific and MTurk samples
Albert, D. A., & Smilek, D. (2023) · Scientific Reports · doi:10.1038/s41598-023-46048-5 · Cited by 166
“Like in previous studies, we also found significant differences in demographic characteristics, namely education and employment in E1 and age and employment in E2, between the MTurk and Prolific samples 40,70,71 .”
ResultDing, Z., Srinivasan, A., Macneil, S., & Chan, J. (2023) · Creativity and Cognition · doi:10.1145/3591196.3593516 · Cited by 35
“We recruited participants from Prolific 6 as Pro-lific participants typically exhibit greater care in completing tasks compared to Amazon MTurk participants [74].”
BackgroundTowards a Reflection in Creative Experience Questionnaire
Ford, C., & Bryan-Kinns, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581077 · Cited by 15
“We use Prolific instead of alternatives because it is academic-focused and its participants might show more interest in creativity-related work [66].”
Metadata
- Authors
- Oppenlaender, J., Milland, K., Visuri, A., Ipeirotis, P., & Hosio, S.
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-14
- OpenAlex
- W3002599298
- Cited by
- 35
- References
- 16
- FWCI
- 6.54
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Creativity on Paid Crowdsourcing Platforms. https://app.cochat.ai/papers/10.1145/3313831.3376677
- The paper
- Oppenlaender, J., Milland, K., Visuri, A., Ipeirotis, P., & Hosio, S. (2020). Creativity on Paid Crowdsourcing Platforms. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376677
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.