Creation of the algorithmic management questionnaire: A six‐phase scale development process
Open accessThe DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Abstract
There is an increasing body of research on algorithmic management (AM), but the field lacks measurement tools to capture workers' experiences of this phenomenon. Based on existing literature, we developed and validated the algorithmic management questionnaire (AMQ) to measure the perceptions of workers regarding their level of exposure to AM. Across three samples (overall n = 1332 gig workers), we show the content, factorial, discriminant, convergent, and predictive validity of the scale. The final 20‐item scale assesses workers' perceived level of exposure to algorithmic: monitoring, goal setting, scheduling, performance rating, and compensation. These dimensions formed a higher order construct assessing overall exposure to algorithmic management, which was found to be, as expected, negatively related to the work characteristics of job autonomy and job complexity and, indirectly, to work engagement. Supplementary analyses revealed that perceptions of exposure to AM reflect the objective presence of AM dimensions beyond individual variations in exposure. Overall, the results suggest the suitability of the AMQ to assess workers' perceived exposure to algorithmic management, which paves the way for further research on the impacts of these rapidly accelerating systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
The job demands-resources model of burnout.
Demerouti, E., Bakker, A. B., Nachreiner, F., & Schaufeli, W. B. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.3.499 · Cited by 13,643
Job demands, job resources, and their relationship with burnout and engagement: a multi‐sample study
Schaufeli, W. B., & Bakker, A. B. (2004) · Journal of Organizational Behavior · doi:10.1002/job.248 · Cited by 9,297
Motivation through the design of work: test of a theory
Hackman, J. R., & Oldham, G. R. (1976) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(76)90016-7 · Cited by 8,962
The Measurement of Work Engagement With a Short Questionnaire: A Cross-National Study
Schaufeli, W. B., Bakker, A. B., & Salanova, M. (2006) · Educational and Psychological Measurement · doi:10.1177/0013164405282471 · Cited by 7,403
Jarvis, C. B., MacKenzie, S. B., & Podsakoff, P. M. (2003) · Journal of Consumer Research · doi:10.1086/376806 · Cited by 5,807
A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires
Hinkin, T. R. (1998) · Organizational Research Methods · doi:10.1177/109442819800100106 · Cited by 4,095
A Review of Scale Development Practices in the Study of Organizations
Hinkin, T. R. (1995) · Journal of Management · doi:10.1177/014920639502100509 · Cited by 3,193
MacKenzie, S. B., Podsakoff, P. M., & Podsakoff, N. P. (2011) · MIS Quarterly · doi:10.2307/23044045 · Cited by 3,165
Morgeson, F. P., & Humphrey, S. E. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.6.1321 · Cited by 2,994
Artificial Intelligence and Management: The Automation–Augmentation Paradox
Raisch, S., & Krakowski, S. (2021) · Academy of Management Review · doi:10.5465/amr.2018.0072 · Cited by 2,755
Algorithms at Work: The New Contested Terrain of Control
Kellogg, K. C., Valentine, M. A., & Christin, A. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0174 · Cited by 2,732
Good Gig, Bad Gig: Autonomy and Algorithmic Control in the Global Gig Economy
Wood, A. J., Graham, M., Lehdonvirta, V., & Hjorth, I. (2019) · Work, Employment and Society · doi:10.1177/0950017018785616 · Cited by 2,209
Artificial Intelligence in Human Resources Management: Challenges and a Path Forward
Tambe, P., Cappelli, P., & Yakubovich, V. (2019) · California Management Review · doi:10.1177/0008125619867910 · Cited by 1,609
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338
Vrontis, D., Christofi, M., Pereira, V., Tarba, S., Makrides, A., & Trichina, E. (2022) · The International Journal of Human Resource Management · doi:10.1080/09585192.2020.1871398 · Cited by 1,134
Duggan, J., Sherman, U., Carbery, R., & McDonnell, A. (2020) · Human Resource Management Journal · doi:10.1111/1748-8583.12258 · Cited by 1,050
Automation, Algorithms, and Beyond: Why Work Design Matters More Than Ever in a Digital World
Parker, S. K., & Grote, G. (2022) · Applied Psychology · doi:10.1111/apps.12241 · Cited by 960
Self-citationLabour process theory and the gig economy
Gandini, A. (2019) · Human Relations · doi:10.1177/0018726718790002 · Cited by 946
Working with Machines: The Impact of Algorithmic and Data-Driven Management on Human Workers
Lee, M. K., Kusbit, D., Metsky, E., & Dabbish, L. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702548 · Cited by 823
Makarius, E. E., Mukherjee, D., Fox, J. D., & Fox, A. K. (2020) · Journal of Business Research · doi:10.1016/j.jbusres.2020.07.045 · Cited by 780
Uberland: How Algorithms Are Rewriting the Rules of Work
Rosenblat, A. (2018) · doi:10.1525/9780520970632 · Cited by 694
Veen, A., Barratt, T., & Goods, C. (2020) · Work, Employment and Society · doi:10.1177/0950017019836911 · Cited by 657
Taxonomic efforts in the description of leader behavior: A synthesis and functional interpretation
Fleishman, E. A., Mumford, M. D., Zaccaro, S. J., Levin, K. Y., Korotkin, A. L., & Hein, M. B. (1991) · The Leadership Quarterly · doi:10.1016/1048-9843(91)90016-u · Cited by 580
Algorithmic Management of Work on Online Labor Platforms: When Matching Meets Control
Möhlmann, M., Zalmanson, L., Henfridsson, O., & Gregory, R. W. (2021) · MIS Quarterly · doi:10.25300/misq/2021/15333 · Cited by 554
Newman, D. T., Fast, N. J., & Harmon, D. J. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.03.008 · Cited by 550
Flexibility in the gig economy: managing time on three online piecework platforms
Lehdonvirta, V. (2018) · New Technology, Work and Employment · doi:10.1111/ntwe.12102 · Cited by 550
Algorithmic management in a work context
Jarrahi, M. H., Newlands, G., Lee, M. K., Wolf, C. T., Kinder, E., & Sutherland, W. (2021) · Big Data & Society · doi:10.1177/20539517211020332 · Cited by 532
Humans and Technology: Forms of Conjoined Agency in Organizations
Murray, A., Rhymer, J., & Sirmon, D. G. (2021) · Academy of Management Review · doi:10.5465/amr.2019.0186 · Cited by 518
Algorithms as work designers: How algorithmic management influences the design of jobs
Parent-Rocheleau, X., & Parker, S. K. (2022) · Human Resource Management Review · doi:10.1016/j.hrmr.2021.100838 · Cited by 512
Self-citationAlgorithmic Control in Platform Food Delivery Work
Griesbach, K., Reich, A., Elliott-Negri, L., & Milkman, R. (2019) · Socius: Sociological Research for a Dynamic World · doi:10.1177/2378023119870041 · Cited by 443
Newlands, G. (2021) · Organization Studies · doi:10.1177/0170840620937900 · Cited by 439
Day, A., Paquet, S., Scott, N., & Hambley, L. (2012) · Journal of Occupational Health Psychology · doi:10.1037/a0029837 · Cited by 436
Understanding and shaping the future of work with self-determination theory
Gagné, M., Parker, S. K., Griffin, M. A., Dunlop, P. D., Knight, C., Klonek, F. E., & Parent-Rocheleau, X. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-022-00056-w · Cited by 434
Self-citationArtificial Intelligence as Augmenting Automation: Implications for Employment
Tschang, F. T., & Almirall, E. (2021) · Academy of Management Perspectives · doi:10.5465/amp.2019.0062 · Cited by 409
Algorithms in practice: Comparing web journalism and criminal justice
Christin, A. (2017) · Big Data & Society · doi:10.1177/2053951717718855 · Cited by 379
Wang, B., Liu, Y., & Parker, S. K. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0127 · Cited by 377
Self-citationThe Invisible Cage: Workers’ Reactivity to Opaque Algorithmic Evaluations
Rahman, H. A. (2021) · Administrative Science Quarterly · doi:10.1177/00018392211010118 · Cited by 375
Between autonomy and control: Strategies of arbitrage in the “on-demand” economy
Shapiro, A. (2018) · New Media & Society · doi:10.1177/1461444817738236 · Cited by 374
“Is your gig any good?” Analysing job quality in the Australian platform-based food-delivery sector
Goods, C., Veen, A., & Barratt, T. (2019) · Journal of Industrial Relations · doi:10.1177/0022185618817069 · Cited by 365
Working for an Algorithm: Power Asymmetries and Agency in Online Work Settings
Curchod, C., Patriotta, G., Cohen, L., & Neysen, N. (2020) · Administrative Science Quarterly · doi:10.1177/0001839219867024 · Cited by 364
Bucher, E. L., Schou, P. K., & Waldkirch, M. (2021) · Organization · doi:10.1177/1350508420961531 · Cited by 349
The Challenges of Algorithm-Based HR Decision-Making for Personal Integrity
Leicht-Deobald, U., Busch, T., Schank, C., Weibel, A., Schafheitle, S., Wildhaber, I., & Kasper, G. (2019) · Journal of Business Ethics · doi:10.1007/s10551-019-04204-w · Cited by 347
Breaking the vicious cycle of algorithmic management: A virtue ethics approach to people analytics
Gal, U., Jensen, T. B., & Stein, M.-K. (2020) · Information and Organization · doi:10.1016/j.infoandorg.2020.100301 · Cited by 342
Langer, M., & Landers, R. N. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106878 · Cited by 338
Rani, U., & Furrer, M. (2021) · Competition & Change · doi:10.1177/1024529420905187 · Cited by 306
A critical review of algorithms in HRM: Definition, theory, and practice
Cheng, M. M., & Hackett, R. D. (2021) · Human Resource Management Review · doi:10.1016/j.hrmr.2019.100698 · Cited by 283
EPM 20/20: A Review, Framework, and Research Agenda for Electronic Performance Monitoring
Ravid, D. M., Tomczak, D. L., White, J. C., & Behrend, T. S. (2020) · Journal of Management · doi:10.1177/0149206319869435 · Cited by 277
Cameron, L. D., & Rahman, H. (2022) · Organization Science · doi:10.1287/orsc.2021.1557 · Cited by 269
Meijerink, J., Boons, M., Keegan, A., & Marler, J. (2021) · The International Journal of Human Resource Management · doi:10.1080/09585192.2021.1925326 · Cited by 267
AI-augmented HRM: Antecedents, assimilation and multilevel consequences
Prikshat, V., Malik, A., & Budhwar, P. (2023) · Human Resource Management Review · doi:10.1016/j.hrmr.2021.100860 · Cited by 200
Controlling space, controlling labour? Contested space in food delivery gig work
Heiland, H. (2021) · New Technology, Work and Employment · doi:10.1111/ntwe.12183 · Cited by 200
Job quality, fair work and gig work: the lived experience of gig workers
Myhill, K., Richards, J., & Sang, K. (2021) · The International Journal of Human Resource Management · doi:10.1080/09585192.2020.1867612 · Cited by 155
Holland, C., Levis, J., Nuggehalli, R., Santilli, B., & Winters, J. (2017) · Interfaces · doi:10.1287/inte.2016.0875 · Cited by 95
Norlander, P., Jukic, N., Varma, A., & Nestorov, S. (2021) · The International Journal of Human Resource Management · doi:10.1080/09585192.2020.1867614 · Cited by 88
Chapter 5 Brave New Digital Work? New Forms of Performance Control in Crowdwork
Gerber, C., & Krzywdzinski, M. (2019) · Research in the Sociology of Work · doi:10.1108/s0277-283320190000033008 · Cited by 84
A smart place to work? Big data systems, labour, control and modern retail stores
Evans, L., & Kitchin, R. (2018) · New Technology, Work and Employment · doi:10.1111/ntwe.12107 · Cited by 61
Schafheitle, S. D., Weibel, A., Ebert, I. L., Kasper, G., Schank, C., & Leicht-Deobald, U. (2020) · Academy of Management Discoveries · doi:10.5465/amd.2019.0002 · Cited by 61
The emotional labor of surveillance: Digital control in Fast fashion retail
Van Oort, M. (2019) · Critical Sociology · doi:10.1177/0896920518778087 · Cited by 60
Moore, S., & Hayes, L. J. B. (2017) · New Technology, Work and Employment · doi:10.1111/ntwe.12087 · Cited by 52
Context Sensitive Technologies and Electronic Employee Monitoring: a Meta-Analytic Review
Backhaus, N. (2019) · 2019 IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii.2019.8700354 · Cited by 35
A Critical Review of Algorithms in HRM: Definition, Theory, and Practice
Cheng, M. M., & Hackett, R. D. (2019) · Academy of Management Proceedings · doi:10.5465/ambpp.2019.18018abstract · Cited by 29
Glaser, A. L. (2021) · Anthropology of Work Review · doi:10.1111/awr.12215 · Cited by 22
Technology introduction on ships: The tension between safety and economic rationality
Bhardwaj, S., Bhattacharya, S., Tang, L., & Howell, K. E. (2019) · Safety Science · doi:10.1016/j.ssci.2019.02.025 · Cited by 19
Cited by
Navigating the maze: the effects of algorithmic management on employee performance
Liu, M., Lan, Y., Liu, Z., Liu, M., & Xia, Y. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03453-z · Cited by 15
Why Does Algorithmic Management Undermine Employee Creativity?: A Perspective Focused on AMO Theory
Li, D., Liu, M., Zhao, Y., Li, Y., Zhang, T., Zhang, W., Xia, D., & Lv, B. (2024) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.340037 · Cited by 12
Zhu, H., Zhao, F., Liu, J., Chen, Y., & Wang, L. (2026) · Human Resource Management · doi:10.1002/hrm.70051 · Cited by 4
Metadata
- Authors
- Parent‐Rocheleau, X., Parker, S. K., Bujold, A., & Gaudet, M.
- Venue
- Human Resource Management
- Publisher
- Wiley
- Published
- 2023-07-16
- Volume
- 63
- Issue
- 1
- Pages
- 25-44
- ISSN
- 0090-4848 (ISSN-L)
- 1099-050X
- OpenAlex
- W4384464462
- Cited by
- 138
- References
- 65
- FWCI
- 60.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.