The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
The Duality of Empowerment and Marginalization in Microtask Crowdsourcing: Giving Voice to the Less Powerful Through Value Sensitive Design1
Abstract
Crowdsourcing (CS) of micro tasks is a relatively new, open source work form enabled by information and communication technologies. While anecdotal evidence of its benefits abounds, our understanding of the phenomenon’s societal consequences remains limited. Drawing on value sensitive design (VSD), we explore microtask CS as perceived by crowd workers, revealing their values as a means of informing the design of CS platforms. Analyzing detailed narratives of 210 crowd workers participating in Amazon’s Mechanical Turk (MTurk), we uncover a set of nine values they share: access, autonomy, fairness, transparency, communication, security, accountability, making an impact, and dignity. We find that these values are implicated in four crowdsourcing structures: compensation, governance, technology, and microtask. Two contrasting perceptions—empowerment and marginalization—coexist, forming a duality of microtask CS. The study contributes to the CS and VSD literatures, heightens awareness of worker marginalization in microtask CS, and offers guidelines for improving CS practice. Specifically, we offer recommendations regarding the ethical use of crowd workers (including for academic research), and call for improving MTurk platform design for greater worker empowerment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hartley, J. (2004) · Essential Guide to Qualitative Methods in Organizational Research · doi:10.4135/9781446280119.n26 · Cited by 15,461
Adams, J. S. (1965) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60108-2 · Cited by 10,818
Design science in information systems research
(2004) · Journal of the Association for Information Systems · doi:10.5555/2017212.2017217 · Cited by 8,254
Development of the Job Diagnostic Survey.
Hackman, J. R., & Oldham, G. R. (1975) · Journal of Applied Psychology · doi:10.1037/h0076546 · Cited by 7,246
A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems1
Klein, H. K., & Myers, M. D. (1999) · MIS Quarterly · doi:10.2307/249410 · Cited by 5,263
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Kittur, A., Nickerson, J. V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., & Horton, J. (2013) · Proceedings of the 2013 conference on Computer supported cooperative work · doi:10.1145/2441776.2441923 · Cited by 1,034 · cited 5×
“Scholars (e.g., Finnerty et al. 2013; Kittur et al. 2013) have evaluated the CS compensation structure (e.”
InfluentialFriedman, B. (1996) · Interactions · doi:10.1145/242485.242493 · Cited by 966 · cited 1×
“To explore worker values that could be foundational in designing CS platforms, we draw on value sensitive design (VSD) (Friedman 1996; Friedman and Khan 2003; Friedman et al. 2006) in analyzing the narratives of MTurk crowd workers.”
MethodBerg, J. M., Wrzesniewski, A., & Dutton, J. E. (2010) · Journal of Organizational Behavior · doi:10.1002/job.645 · Cited by 890
Viewpoint: choosing appropriate information systems research methodologies
Galliers, R. D., & Land, F. F. (1987) · Communications of the ACM · doi:10.1145/32206.315753 · Cited by 181
Self-citation
Research works printed but not linked: 16
- Arthur, 1996, The Boundaryless Career: A New Employment Principle for a New Organizational Era, 10.1093/oso/9780195100143.001.0001 · DOI · Paper page
- Cowherd, 1992, Product Quality and Pay Equity Between Lower-Level Employees and Top Management: An Investigation of Distributive Justice Theory, Administrative Science Quarterly, 37, 302, 10.2307/2393226 · DOI · Paper page
- Friedman, 2008, Value Sensitive Design and Information Systems, 69, 10.1002/9780470281819.ch4 · DOI · Paper page
- Friedman, 2000, Trust Online, Communications of the ACM, 43, 34, 10.1145/355112.355120 · DOI · Paper page
- Gilliland, 2001, Theoretical and Cultural Perspectives on Organizational Justice
- Gregor, 2013, Positioning and Presenting Design Science Research for Maximum Impact, MIS Quarterly, 37, 337, 10.25300/MISQ/2013/37.2.01 · DOI · Paper page
- Hackman, 1976, Motivation Through the Design of Work: Test of a Theory, Organizational Behavior and Human Performance, 16, 250, 10.1016/0030-5073(76)90016-7 · DOI · Paper page
- Hakim, 2000, Work-Lifestyle Choices in the 21st Century: Preference Theory, 10.1093/oso/9780199242092.001.0001 · DOI · Paper page
- Laguna, 2013, Business Process Modeling: Simulation and Design, 10.1201/b14763 · DOI
- Millett, 2001, Cookies and Web Browser Design: Toward Realizing Informed Consent Online, 10.1145/365024.365034 · DOI
- Muffels, 2005, Labour Market Mobility and Empowerment Patterns
- Muffels, 2005, Job Mobility and Employment Patterns across European Welfare States: Is There a ‘Trade-Off’ or a ‘Double Bind” Between Flexibility and Security?
- Oldham, 2010, Not What it Was and Not What it Will Be: The Future of Jo Design Research, Journal of Organizational Behavior, 31, 453, 10.1002/job.678 · DOI · Paper page
- Rawls, 1971, A Theory of Justice, 10.4159/9780674042605 · DOI · Paper page
- Taylor, 1911, Principles of Scientific Management
- Wrzesniewski, 2001, Crafting a Job: Revisioning Employees as Active Crafters of Their Work, The Academy of Management Review, 26, 179, 10.2307/259118 · DOI · Paper page
Cited by
Understanding digital transformation: A review and a research agenda
Vial, G. (2019) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2019.01.003 · Cited by 7,337
The sharing economy and digital platforms: A review and research agenda
Sutherland, W., & Jarrahi, M. H. (2018) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2018.07.004 · Cited by 725
Algorithmic Management of Work on Online Labor Platforms: When Matching Meets Control
Möhlmann, M., Zalmanson, L., Henfridsson, O., & Gregory, R. W. (2021) · MIS Quarterly · doi:10.25300/misq/2021/15333 · Cited by 554
Kuhn, K. M., & Maleki, A. (2017) · Academy of Management Perspectives · doi:10.5465/amp.2015.0111 · Cited by 430
A Survey of Value Sensitive Design Methods
Friedman, B., Hendry, D. G., & Borning, A. (2017) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000015 · Cited by 389
Metadata
- Authors
- Deng, X. (., Joshi, K. D., & Galliers, R. D.
- Venue
- MIS Quarterly
- Publisher
- MIS Quarterly
- Published
- 2016-06-01
- Volume
- 40
- Issue
- 2
- Pages
- 279-302
- ISSN
- 0276-7783 (ISSN-L)
- 2162-9730
- OpenAlex
- W2412666979
- Cited by
- 303
- References
- 12
- FWCI
- 29.05
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
