CoChat

The Illusion of Artificial Inclusion

Open access
Agnew, William, Bergman, A. Stevie, Chien, Jennifer, Díaz, Mark, El-Sayed, Seliem, Pittman, Jaylen, Mohamed, Shakir, McKee, Kevin R. · 2024 · Proceedings of the CHI Conference on Human Factors in Computing Systems
Cited by 58

The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.

Abstract

Human participants play a central role in the development of modern artificial intelligence (AI) technology, in psychological science, and in user research. Recent advances in generative AI have attracted growing interest to the possibility of replacing human participants in these domains with AI surrogates. We survey several such “substitution proposals” to better understand the arguments for and against substituting human participants with modern generative AI. Our scoping review indicates that the recent wave of these proposals is motivated by goals such as reducing the costs of research and development work and increasing the diversity of collected data. However, these proposals ignore and ultimately conflict with foundational values of work with human participants: representation, inclusion, and understanding. This paper critically examines the principles and goals underlying human participation to help chart out paths for future work that truly centers and empowers participants.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3613904.3642703

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 43 references

Research works printed but not linked: 94

  1. Daron Acemoğlu and James A. Robinson. 2016. Paths to inclusive political institutions. Economic History of Warfare and State Formation (2016), 3–50.
  2. William Agnew, Kevin R. McKee, Iason Gabriel, Jackie Kay, William Isaac, A. Stevie Bergman, Seliem El-Sayed, and Shakir Mohamed. 2023. Technologies of resistance to AI. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1–13.
  3. 10.1177/17456916221105213 · DOI
  4. 10.1177/0010414017720702 · DOI · Paper page
  5. 10.1080/01944366908977225 · DOI · Paper page
  6. Neda Atanasoski and Kalindi Vora. 2019. Surrogate humanity: Race, robots, and the politics of technological futures. Duke University Press.
  7. Mohammad Atari, Mona J. Xue, Peter S. Park, Damián Blasi, and Joseph Henrich. 2023. Which humans?PsyArXiv (2023).
  8. Rowland Atkinson and John Flint. 2001. Accessing hidden and hard-to-reach populations: Snowball research strategies. Social Research Update 33, 1 (2001), 1–4.
  9. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, 2022. Constitutional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073 (2022). · arXiv
  10. 10.3102/0013189X20927363 · DOI · Paper page
  11. 10.1016/S0020-7373(87)80047-0 · DOI · Paper page
  12. Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 5185–5198.
  13. Bernard Berelson. 1952. Content analysis in communication research. Free Press.
  14. 10.18653/v1/2022.findings-acl.31 · DOI
  15. 10.1145/3593013.3594019 · DOI
  16. 10.1080/13600869.1986.9966228 · doi:10.1080/13600869.1986.9966228
  17. 10.1016/S0953-5438(97)00032-5 · DOI · Paper page
  18. Allan M. Brandt. 1978. Racism and research: The case of the Tuskegee Syphilis Study. Hastings Center Report (1978), 21–29.
  19. 10.1145/3404983.3405506 · DOI
  20. 10.1145/3544549.3582749 · DOI
  21. 10.1145/3442188.3445872 · DOI
  22. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021). · arXiv
  23. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, 2022. PaLM: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 (2022). · arXiv
  24. Luigina Ciolfi, Myriam Lewkowicz, and Kjeld Schmidt. 2023. Computer-supported cooperative work. In Handbook of Human-Computer Interaction. Springer, 1–26.
  25. Richard W. Coan. 1979. Psychologists: Personal and theoretical pathways. Irvington Publishers.
  26. 10.1146/annurev.psych.49.1.559 · DOI · Paper page
  27. 10.1093/cdj/bsn010 · DOI · Paper page
  28. Critical Platform Studies Group, Lilly Irani, Niloufar Salehi, Joyojeet Pal, Andrés Monroy-Hernández, Elizabeth Churchill, and Sneha Narayan. 2019. Patron or poison? Industry funding of HCI research. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing. 111–115.
  29. Molly Crockett and Lisa Messeri. 2023. Should large language models replace human participants?PsyArXiv (2023).
  30. Robert A. Dahl. 2008. Polyarchy: Participation and opposition. Yale University Press.
  31. Kurt Danziger. 1994. Constructing the subject: Historical origins of psychological research. Cambridge University Press.
  32. Danica Dillion, Niket Tandon, Yuling Gu, and Kurt Gray. 2023. Can AI language models replace human participants?Trends in Cognitive Sciences (2023).
  33. 10.1177/1463499610370517 · DOI · Paper page
  34. Orlando Fals-Borda and Muhammad Anisur Rahman. 1991. Action and knowledge: Breaking the monopoly with participatory action-research. Intermediate Technology Publications.
  35. 10.1177/20539517221082027 · DOI · Paper page
  36. Marco A. Gerosa, Bianca Trinkenreich, Igor Steinmacher, and Anita Sarma. 2023. Can AI serve as a substitute for human subjects in software engineering research?arXiv preprint arXiv:2311.11081 (2023). · arXiv
  37. 10.1007/s10606-021-09422-3 · DOI
  38. 10.1145/3593013.3594071 · DOI · Paper page
  39. 10.1609/aimag.v30i4.2271 · DOI · Paper page
  40. Jonathan Grudin. 2022. From tool to partner: The evolution of human-computer interaction. Springer Nature.
  41. 10.1145/3544548.3580688 · DOI · Paper page
  42. David Heckerman, David Maxwell Chickering, Christopher Meek, Robert Rounthwaite, and Carl Kadie. 2000. Dependency networks for inference, collaborative filtering, and data visualization. Journal of Machine Learning Research 1, Oct (2000), 49–75.
  43. Tom Heyman and Geert Heyman. 2023. The impact of ChatGPT on human data collection: A case study involving typicality norming data. Behavior Research Methods (2023), 1–8.
  44. Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, 2022. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303 (2022). · arXiv
  45. John J. Horton. 2023. Large language models as simulated economic agents: What can we learn from homo silicus?Technical Report. National Bureau of Economic Research.
  46. @infoxiao. 2023. A trend I see at #CHI23 is HCI researchers beginning to use LLMs to create synthetic user studies. E.g. https://dl.acm.org/doi/10.1145/3544548.3580688. While it is convenient I do worry about the potential representational harms posed by such trend – experiences of minorities being erased.https://web.archive.org/web/20240113081958/https://nitter.net/infoxiao/status/1651160544307544064. · DOI · Paper page
  47. 10.1177/1948550617691100 · DOI · Paper page
  48. @jacyanthis. 2023. Here’s the latest iteration of the “techniques like RLHF are just putting superficial smiley faces on an opaque, alien, Shoggoth intelligence that we have no real control over” AI safety meme. Artist: Anna Husfeldt, CC-BY SA 3.0. https://web.archive.org/web/20230306110137/https://twitter.com/jacyanthis/status/1631291175381475331.
  49. Michelle K. Jamieson, Gisela H. Govaart, and Madeleine Pownall. 2023. Reflexivity in quantitative research: A rationale and beginner’s guide. Social and Personality Psychology Compass (2023), e12735.
  50. 10.1145/3593013.3594060 · DOI
  51. J. Jones. 2018. Facebook users’ privacy concerns up since 2011. The Gallup Organization.
  52. 10.1016/j.jretconser.2023.103580 · DOI · Paper page
  53. 10.1080/00049180500325694 · DOI · Paper page
  54. Steven Levy. 1992. Artificial life: The quest for a new creation. Random House Inc.
  55. 10.1177/20539517211047734 · DOI · Paper page
  56. 10.1162/daed_a_01905 · DOI · Paper page
  57. 10.1111/j.1467-9280.2009.02453.x · DOI · Paper page
  58. 10.1353/tech.2010.0009 · DOI · Paper page
  59. 10.1080/10720537.2019.1635924 · DOI
  60. Renate Mayntz. 2005. Forschungsmethoden und erkenntnispotential: Natur-und sozialwissenschaften im vergleich. Max-Planck-Institut für Gesellschaftsforschung.
  61. Kevin R. McKee. 2023. Human participants in AI research: Ethics and transparency in practice. arXiv preprint arXiv:2311.01254 (2023). · arXiv
  62. 10.1037/0003-066X.38.4.379 · DOI · Paper page
  63. 10.21236/ADA125318 · DOI
  64. 10.1002/ejsp.2724 · DOI
  65. Matt Novak. 2023. That viral image of Pope Francis wearing a white puffer coat is totally fake. Forbes (2023). https://web.archive.org/web/20230914181410/https://www.forbes.com/sites/mattnovak/2023/03/26/that-viral-image-of-pope-francis-wearing-a-white-puffer-coat-is-totally-fake/
  66. OpenAI. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774 (2023). · arXiv
  67. 10.18653/v1/2021.acl-long.329 · DOI · Paper page
  68. Amandalynne Paullada, Inioluwa Deborah Raji, Emily M. Bender, Emily Denton, and Alex Hanna. 2021. Data and its (dis)contents: A survey of dataset development and use in machine learning research. Patterns 2, 11 (2021).
  69. 10.1177/0739456X11410979 · DOI · Paper page
  70. Julian Reiss and Jan Sprenger. 2014. Scientific objectivity. In The Stanford Encyclopedia of Philosophy. Zalta, Ed.
  71. Jennifer Rhee. 2018. The robotic imaginary: The human and the price of dehumanized labor. University of Minnesota Press.
  72. David K. Robinson. 2001. Reaction-time experiments in Wundt’s institute and beyond. In Wilhelm Wundt in History: The Making of a Scientific Psychology. Springer, 161–204.
  73. Esther Rolf, Theodora T. Worledge, Benjamin Recht, and Michael Jordan. 2021. Representation matters: Assessing the importance of subgroup allocations in training data. In Proceedings of the 38th International Conference on Machine Learning. Proceedings of Machine Learning Research, 9040–9051.
  74. Kevin Roose. 2023. A conversation with Bing’s chatbot left me deeply unsettled. The New York Times (2023). https://web.archive.org/web/20230914044532/https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html
  75. 10.1145/3466160 · DOI
  76. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect?arXiv preprint arXiv:2303.17548 (2023). · arXiv
  77. 10.1353/hpu.0.0323 · DOI · Paper page
  78. 10.1016/0004-3702(78)90012-7 · DOI
  79. 10.1145/3498366.3505816 · DOI · Paper page
  80. Murray Shanahan. 2022. Talking about large language models. arXiv preprint arXiv:2212.03551 (2022). · arXiv
  81. 10.18653/v1/2021.naacl-main.295 · DOI · Paper page
  82. Åsne Stige, Efpraxia D. Zamani, Patrick Mikalef, and Yuzhen Zhu. 2023. Artificial intelligence (AI) for user experience (UX) design: A systematic literature review and future research agenda. Information Technology & People (2023).
  83. Derek Thompson. 2022. Breakthroughs of the year. The Atlantic (2022). https://web.archive.org/web/20230421060355/https://www.theatlantic.com/newsletters/archive/2022/12/technology-medicine-law-ai-10-breakthroughs-2022/672390/
  84. Nitasha Tiku. 2022. The Google engineer who thinks the company’s AI has come to life. The Washington Post (2022). https://web.archive.org/web/20230826183724/https://www.washingtonpost.com/technology/2022/06/11/google-ai-lamda-blake-lemoine/
  85. 10.1007/BF00430638 · doi:10.1007/bf00430638
  86. James Vincent. 2022. All these images were generated by Google’s latest text-to-image AI. The Verge (2022). https://web.archive.org/web/20230725102931/https://www.theverge.com/2022/5/24/23139297/google-imagen-text-to-image-ai-system-examples-paper
  87. 10.1145/2470654.2470716 · DOI · Paper page
  88. 10.18653/v1/2021.findings-emnlp.354 · DOI · Paper page
  89. Max Weber, A. M. Henderson, and Talcott Parsons. 1947. The theory of social and economic organization. Free Press.
  90. Benjamin Weiser. 2023. Here’s what happens when your lawyer uses ChatGPT. The New York Times (2023). https://web.archive.org/web/20230913041650/https://www.nytimes.com/2023/05/27/nyregion/avianca-airline-lawsuit-chatgpt.html
  91. Blaise Agüera y Arcas. 2022. Do large language models understand us?Daedalus 151, 2 (2022), 183–197.
  92. 10.1145/2858036.2858523 · DOI · Paper page
  93. 10.1177/016224399101600106 · DOI · Paper page
  94. 10.1145/3491101.3516506 · DOI · Paper page

Cited by

See all 58

Metadata

Authors
Agnew, W., Bergman, A. S., Chien, J., Díaz, M., El-Sayed, S., Pittman, J., Mohamed, S., & McKee, K. R.
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-12
OpenAlex
W4396833703
Cited by
58
References
43
FWCI
27.19
Type
conference paper
Language
en
Open access
hybrid

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Illusion of Artificial Inclusion. https://app.cochat.ai/papers/10.1145/3613904.3642703
The paper
Agnew, W., Bergman, A. S., Chien, J., Díaz, M., El-Sayed, S., Pittman, J., Mohamed, S., & McKee, K. R. (2024). The Illusion of Artificial Inclusion. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642703

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.