CoChat
Sign Up
PreprintCited by 34Social Robot Interaction and HRIHuman-Automation Interaction and SafetyPain Management and Placebo Effect

"AI enhances our performance, I have no doubt this one will do the same": The Placebo effect is robust to negative descriptions of AI

Open access
Kloft, Agnes M., Welsch, Robin, Kosch, Thomas, Villa, Steeven·2023·arXiv
DOI 10.48550/arxiv.2309.16606arXiv 2309.16606
Open this paper in CoChat — free account

Abstract

Heightened AI expectations facilitate performance in human-AI interactions through placebo effects. While lowering expectations to control for placebo effects is advisable, overly negative expectations could induce nocebo effects. In a letter discrimination task, we informed participants that an AI would either increase or decrease their performance by adapting the interface, but in reality, no AI was present in any condition. A Bayesian analysis showed that participants had high expectations and performed descriptively better irrespective of the AI description when a sham-AI was present. Using cognitive modeling, we could trace this advantage back to participants gathering more information. A replication study verified that negative AI descriptions do not alter expectations, suggesting that performance expectations with AI are biased and robust to negative verbal descriptions. We discuss the impact of user expectations on AI interactions and evaluation and provide a behavioral placebo marker for human-AI interaction

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2309.16606

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • People tend to over-rely and over-trust AI systems.

    “other research shows people tend to over-rely and over-trust AI systems [3, 28, 31, 50].”

    From Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences (2024) · cites this paper as [28]

    The Discussion (p. 16) reports that participants expected AI to improve their performance even after being told it would make the task harder, and the Qualitative Results (p. 15) describe trust in AI’s capabilities. These findings support unusually strong trust in this study’s setting, but the paper does not establish that people generally over-rely on AI systems. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 57

  1. [1][1] Lauren Y Atlas. 2021. A social affective neuroscience lens on placebo analgesia. Trends in Cognitive Sciences 25, 11 (Nov. 2021), 992-1005. https://doi.org/10.1016/j.tics.2021.07.016 - · DOI
  2. [2][2] Ebrahim Babaei, Benjamin Tag, Tilman Dingler, and Eduardo Velloso. 2021. A Critique of Electrodermal Activity Practices at CHI. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (Chi '21) . Association for Computing Machinery, New York, NY, USA, Article 177, 14 pages. https://doi.org/10.1145/ 3411764.3445370 -
  3. [3][3] Jascha Bareis and Christian Katzenbach. 2022. Talking AI into being: The narratives and imaginaries of national AI strategies and their performative politics. Science, Technology, & Human Values 47, 5 (May 2022), 855-881. https: //doi.org/10.1177/01622439211030007 Not for distribution. Unpublished working draft. · DOI
  4. [4][4] Ernest Edward Beckham. 1989. Improvement after evaluation in psychotherapy of depression: evidence of a placebo effect? Journal of clinical psychology 45, 6 (Nov. 1989), 945-950. https://doi.org/10.1002/1097-4679(198911)45:6<945:: aid-jclp2270450620>3.0.co;2-2 · DOI
  5. [5][5] Henry K. Beecher. 1955. The powerful placebo. Journal of the American Medical Association 159, 17 (Dec. 1955), 1602-1606. https://doi.org/10.1001/jama.1955.02960340022006 arXiv:https://jamanetwork.com/journals/jama/articlepdf/303530/jama\_159\_17\_006.pdf · DOI
  6. [6][6] Christopher J Beedie, Damian A Coleman, and Abigail J Foad. 2007. Positive and negative placebo effects resulting from the deceptive administration of an ergogenic aid. International journal of sport nutrition and exercise metabolism 17, 3 (June 2007), 259-269. https://doi.org/10.1123/ijsnem.17.3.259 · DOI
  7. [7][7] Walter R. Boot, Daniel J. Simons, Cary Stothart, and Cassie Stutts. 2013. The Pervasive Problem With Placebos in Psychology: Why Active Control Groups Are Not Sufficient to Rule Out Placebo Effects. Perspectives on Psychological Science 8, 4 (2013), 445-454. https://doi.org/10.1177/1745691613491271 · DOI · Paper page
  8. [8][8] Paolo Bory. 2019. Deep new: The shifting narratives of artificial intelligence from Deep Blue to AlphaGo. Convergence 25, 4 (Feb. 2019), 627-642. https://doi.org/10.1177/1354856519829679 · DOI
  9. [10][10] Paul-Christian Bürkner. 2017. brms: An R package for Bayesian multilevel models using Stan. Journal of statistical software 80, 1 (Aug. 2017), 1-28. https://doi.org/10.18637/jss.v080.i01 · DOI · Paper page
  10. [11][11] Astrid Carolus, Martin Koch, Samantha Straka, Marc Latoschik, and Carolin Wienrich. 2023. MAILS - Meta AI Literacy Scale: Development and Testing of an AI Literacy Questionnaire Based on Well-Founded Competency Models and Psychological Change- and Meta-Competencies. (Feb. 2023). arXiv:2302.09319 [cs.AI] · arXiv
  11. [12][12] Bob Carpenter, Andrew Gelman, Matthew D. Hoffman, Daniel Lee, Ben Goodrich, Michael Betancourt, Marcus Brubaker, Jiqiang Guo, Peter Li, and Allen Riddell. 2017. Stan: A Probabilistic Programming Language. Journal of Statistical Software 76, 1 (Jan. 2017), 1-32. https://doi.org/10.18637/jss.v076.i01 · DOI · Paper page
  12. [13][13] Stephen Cave, Kate Coughlan, and Kanta Dihal. 2019. "Scary Robots": Examining Public Responses to AI. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (Honolulu, HI, USA) (Aies '19) . Association for Computing Machinery, New York, NY, USA, 331-337. https://doi.org/10.1145/3306618.3314232 · DOI
  13. [14][14] Stephen Cave and Kanta Dihal. 2019. Hopes and fears for intelligent machines in fiction and reality. Nature Machine Intelligence 1 (Feb. 2019), 74-78. https://doi.org/10.1038/s42256-019-0020-9 · DOI
  14. [15][15] Javier Corredor, Jorge Sofrony, and Angelika Peer. 2017. Decision-Making Model for Adaptive Impedance Control of Teleoperation Systems. Institute of Electrical and Electronics Engineers (IEEE) Transactions on Haptics 10, 1 (Jan. 2017), 5-16. https://doi.org/10.1109/toh.2016.2581807 - · DOI
  15. [17][17] Nicola Dell, Vidya Vaidyanathan, Indrani Medhi, Edward Cutrell, and William Thies. 2012. " Yours is better!" participant response bias in HCI. In Proceedings of the sigchi conference on human factors in computing systems . 1321-1330. -
  16. [18][18] Alena Denisova and Paul Cairns. 2015. The Placebo Effect in Digital Games: Phantom Perception of Adaptive Artificial Intelligence. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (London, United Kingdom) (Chi Play '15) . Association for Computing Machinery, New York, NY, USA, 23-33. https://doi.org/10.1145/ 2793107.2793109 -
  17. [19][19] Alena Denisova and Eliott Cook. 2019. Power-Ups in Digital Games: The Rewarding Effect of Phantom Game Elements on Player Experience. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Barcelona, Spain) (Chi Play '19) . Association for Computing Machinery, New York, NY, USA, 161-168. https: //doi.org/10.1145/3311350.3347173 - · DOI
  18. [20][20] Nico J. Diederich and Christopher G. Goetz. 2008. The placebo treatments in neurosciences. Neurology 71, 9 (Aug. 2008), 677-684. https://doi.org/10.1212/01.wnl.0000324635.49971.3d - · DOI
  19. [21][21] Alan Dix. 2022. Bayesian statistics. In Bayesian Methods for Interaction and Design , John H. Williamson, Antti Oulasvirta, Per Ola Kristensson, and Nikola Banovic (Eds.). Cambridge University Press, 81-114. https://doi.org/10. 1017/9781108874830.004 -
  20. [22][22] Andrea Ferrario, Michele Loi, and Eleonora Viganò. 2019. In AI We Trust Incrementally: a Multi-layer Model of Trust to Analyze Human-Artificial Intelligence Interactions. Philosophy and Technology 33 (2019), 523-539. -
  21. [23][23] Kraig Finstad. 2006. The System Usability Scale and Non-Native English Speakers. Journal of User Experience 1, 4 (Aug. 2006), 185-188. -
  22. [24][24] Andreas Fügener, Jörn Grahl, Alok Gupta, and Wolfgang Ketter. 2022. Cognitive Challenges in Human-Artificial Intelligence Collaboration: Investigating the Path Toward Productive Delegation. Information Systems Research 33, 2 (June 2022), 678-696. https://doi.org/10.1287/isre.2021.1079 arXiv:https://doi.org/10.1287/isre.2021.1079 - · DOI
  23. [25][25] Andrew L Geers, Shane Close, Fawn C Caplandies, Charles L Vogel, Ashley B Murray, Yopina Pertiwi, Ian M Handley, and Lene Vase. 2019. Testing a positive-affect induction to reduce verbally induced nocebo hyperalgesia in an experimental pain paradigm. Pain 160, 10 (Oct. 2019), 2290-2297. https://doi.org/10.1097/j.pain.0000000000001618 - · DOI
  24. [26][26] Andrew Gelman, Donald B Rubin, et al. 1992. Inference from iterative simulation using multiple sequences. Statistical science 7, 4 (1992), 457-472. -
  25. [27][27] Friedrich M Götz, Samuel D Gosling, and Peter J Rentfrow. 2022. Small effects: The indispensable foundation for a cumulative psychological science. Perspectives on Psychological Science 17, 1 (2022), 205-215. -
  26. [28][28] José Guerreiro, Raúl Martins, Hugo Silva, André Lourenço, and Ana Fred. 2013. BITalino - A Multimodal Platform for Physiological Computing. In Proceedings of the 10th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO . 500-506. https://doi.org/10.5220/0004594105000506 - · DOI
  27. [29][29] David Halbhuber, Maximilian Schlenczek, Johanna Bogon, and Niels Henze. 2022. Better Be Quiet about It! The Effects of Phantom Latency on Experienced First-Person Shooter Players. In Proceedings of the 21st International Conference on Mobile and Ubiquitous Multimedia (Lisbon, Portugal) (Mum '22) . Association for Computing Machinery, New York, NY, USA, 172-181. https://doi.org/10.1145/3568444.3568448 - · DOI
  28. [30][30] Sandra G. Hart. 2006. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50, 9 (Oct. 2006), 904-908. https://doi.org/10.1177/154193120605000909 - · DOI · Paper page
  29. [31][31] Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In Human Mental Workload , Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, Vol. 52. North-Holland, 139-183. https://doi.org/10.1016/S0166-4115(08)62386-9 - · DOI · Paper page
  30. [33][33] Isabella Hermann. 2020. Beware of fictional AI narratives. Nature Machine Intelligence 2, 11 (Oct. 2020), 654-654. https://doi.org/10.1038/s42256-020-00256-0 - · DOI
  31. [34][34] Herbert Hoijtink and Rens van de Schoot. 2018. Testing small variance priors using prior-posterior predictive p values. Psychological Methods 23, 3 (2018), 561-569. https://doi.org/10.1037/met0000131 - · DOI
  32. [35][35] Asbjørn Hrõbjartsson and Peter Christian Gøtzsche. 2001. Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment. The New England journal of medicine 344 21 (May 2001), 1594-602. https: //doi.org/10.1056/nejm200105243442106 - · DOI
  33. [36][36] Jie Huang, Wenhua Wu, Zhenyi Zhang, and Yutao Chen. 2020. A human decision-making behavior model for humanrobot interaction in multi-robot systems. Institute of Electrical and Electronics Engineers (IEEE) Access 8 (Nov. 2020), 197853-197862. https://doi.org/10.1109/access.2020.3035348 - · DOI
  34. [37][37] Philip Hurst, Lieke Schipof-Godart, Attila Szabo, John Raglin, Florentina Hettinga, Bart Roelands, Andrew Lane, Abby Foad, Damian Coleman, and Chris Beedie. 2020. The Placebo and Nocebo effect on sports performance: A systematic review. European Journal of Sport Science 20, 3 (Aug. 2020), 279-292. https://doi.org/10.1080/17461391.2019.1655098 Not for distribution. - · DOI
  35. [38][38] Jiun-Yin Jian, Ann M. Bisantz, and Colin G. Drury. 2000. Foundations for an Empirically Determined Scale of Trust in Automated Systems. International Journal of Cognitive Ergonomics 4, 1 (March 2000), 53-71. https://doi.org/10.1207/ S15327566IJCE0401\_04 Unpublished working draft. -
  36. [39][39] Matthew Kay, Gregory L. Nelson, and Eric B. Hekler. 2016. Researcher-Centered Design of Statistics: Why Bayesian Statistics Better Fit the Culture and Incentives of HCI. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (Chi '16) . Association for Computing Machinery, New York, NY, USA, 4521-4532. https://doi.org/10.1145/2858036.2858465 - · DOI · Paper page
  37. [40][40] William R. King and Jun He. 2006. A meta-analysis of the technology acceptance model. Information & Management 43, 6 (Sept. 2006), 740-755. https://doi.org/10.1016/j.im.2006.05.003 - · DOI · Paper page
  38. [41][41] Thomas Kosch, Jakob Karolus, Johannes Zagermann, Harald Reiterer, Albrecht Schmidt, and Paweł W. Woźniak. 2023. A Survey on Measuring Cognitive Workload in Human-Computer Interaction. ACM Comput. Surv. 55, 13s, Article 283 (July 2023), 39 pages. https://doi.org/10.1145/3582272 - · DOI · Paper page
  39. [42][42] Thomas Kosch, Robin Welsch, Lewis Chuang, and Albrecht Schmidt. 2023. The Placebo Effect of Artificial Intelligence in Human-Computer Interaction. ACM Trans. Comput.-Hum. Interact. 29, 6, Article 56 (Jan. 2023), 32 pages. https: //doi.org/10.1145/3529225 - · DOI · Paper page
  40. [43][43] Louis Lasagna, Frederick Mosteller, John M. von Felsinger, and Henry K. Beecher. 1954. A study of the placebo response. The American Journal of Medicine 16, 6 (June 1954), 770-779. https://doi.org/10.1016/0002-9343(54)90441-6 - · DOI
  41. [44][44] Jonathan Lazar, Julio Abascal, Simone Barbosa, Jeremy Barksdale, Batya Friedman, Jens Grossklags, Jan Gulliksen, Jeff Johnson, Tom McEwan, Loïc Martínez-Normand, Wibke Michalk, Janice Tsai, Gerrit van der Veer, Hans von Axelson, Ake Walldius, Gill Whitney, Marco Winckler, Volker Wulf, Elizabeth F. Churchill, Lorrie Cranor, Janet Davis, Alan Hedge, Harry Hochheiser, Juan Pablo Hourcade, Clayton Lewis, Lisa Nathan, Fabio Paterno, Blake Reid, Whitney Quesenbery, Ted Selker, and Brian Wentz. 2016. Human-Computer Interaction and International Public Policymaking: A Framework for Understanding and Taking Future Actions. Foundations and Trends in Human-Computer Interaction 9, 2 (May 2016), 69-149. https://doi.org/10.1561/1100000062 - · DOI
  42. [45][45] Byungjoo Lee, Sunjun Kim, Antti Oulasvirta, Jong-In Lee, and Eunji Park. 2018. Moving Target Selection: A Cue Integration Model. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (Chi '18) . Association for Computing Machinery, New York, NY, USA, 1-12. https://doi.org/10.1145/3173574. 3173804 - · DOI
  43. [46][46] Douglas G. Lee, Jean Daunizeau, and Giovanni Pezzulo. 2021. Evidence or Confidence: What Is Really Monitored during a Decision? Psychonomic Bulletin & Review (March 2021), 1-20. https://doi.org/10.3758/s13423-023-02255-9 - · DOI
  44. [47][47] Veronika Lerche and Andreas Voss. 2018. Speed-accuracy manipulations and diffusion modeling: Lack of discriminant validity of the manipulation or of the parameter estimates? Behavior Research Methods 50 (March 2018), 2568-2585. https://doi.org/10.3758/s13428-018-1034-7 - · DOI
  45. [48][48] Veronika Lerche, Andreas Voss, and Markus Nagler. 2017. How many trials are required for parameter estimation in diffusion modeling? A comparison of different optimization criteria. Behavior Research Methods 49 (April 2017), 513-537. https://doi.org/10.3758/s13428-016-0740-2 - · DOI
  46. [49][49] Bingjie Liu. 2021. In AI We Trust? Effects of Agency Locus and Transparency on Uncertainty Reduction in Human-AI Interaction. Journal of Computer-Mediated Communication 26, 6 (09 2021), 384-402. https://doi.org/10.1093/jcmc/ zmab013 arXiv:https://academic.oup.com/jcmc/article-pdf/26/6/384/41139653/zmab013.pdf - · DOI
  47. [50][50] Yaxin Liu and Stella F. Lourenco. 2022. Drift diffusion modeling informs how affective factors affect visuospatial decision making. Journal of Vision 22 (Dec. 2022), 3394. https://doi.org/10.1167/jov.22.14.3394 - · DOI
  48. [51][51] Dominique Makowski, Tam Pham, Zen J. Lau, Jan C. Brammer, François Lespinasse, Hung Pham, Christopher Schölzel, and S. H. Annabel Chen. 2021. NeuroKit2: A Python toolbox for neurophysiological signal processing. Behavior Research Methods 53 (Feb. 2021), 1689-1696. https://doi.org/10.3758/s13428-020-01516-y - · DOI
  49. [52][52] Christian Meurisch, Cristina A. Mihale-Wilson, Adrian Hawlitschek, Florian Giger, Florian Müller, Oliver Hinz, and Max Mühlhäuser. 2020. Exploring User Expectations of Proactive AI Systems. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 4, Article 146 (Dec. 2020), 22 pages. https://doi.org/10.1145/3432193 - · DOI
  50. [53][53] Guy Montgomery and Irving Kirsch. 1996. Mechanisms of Placebo Pain Reduction: An Empirical Investigation. Psychological Science 7 (May 1996), 174-176. https://doi.org/10.1111/j.1467-9280.1996.tb00352.x - · DOI
  51. [54][54] Michael D. Nunez, Joachim Vandekerckhove, and Ramesh Srinivasan. 2017. How attention influences perceptual decision making: Single-trial EEG correlates of drift-diffusion model parameters. Journal of Mathematical Psychology 76 (Feb. 2017), 117-130. https://doi.org/10.1016/j.jmp.2016.03.003 - · DOI
  52. [55][55] Antti Oulasvirta, Jussi P. P. Jokinen, and Andrew Howes. 2022. Computational Rationality as a Theory of Interaction. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI '22) . Association for Computing Machinery, New York, NY, USA, Article 359, 14 pages. https://doi.org/10.1145/3491102. 3517739 Not for distribution. - · DOI
  53. [56][56] Hyanghee Park, Daehwan Ahn, Kartik Hosanagar, and Joonhwan Lee. 2021. Human-AI Interaction in Human Resource Management: Understanding Why Employees Resist Algorithmic Evaluation at Workplaces and How to Mitigate Burdens. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (Chi '21) . Association for Computing Machinery, New York, NY, USA, Article 154, 15 pages. https://doi.org/10.1145/3411764. 3445304 Unpublished working draft. - · DOI · Paper page
  54. [57][57] Deborah A. Prentice and Dale T. Miller. 2016. When small effects are impressive. , 99-105 pages. https://doi.org/10. 1037/14805-006 -
  55. [58][58] Donald D. Price, Damien G. Finniss, and Fabrizio Benedetti. 2008. A Comprehensive Review of the Placebo Effect: Recent Advances and Current Thought. Annual Review of Psychology 59 (Jan. 2008), 565-590. https://doi.org/10.1146/ annurev.psych.59.113006.095941 -
  56. [59][59] Zoe A. Purcell, Mengchen Dong, Anne-Marie Nussberger, Nils Köbis, and Maurice Jakesch. 2023. Fears about AI-mediated communication are grounded in different expectations for one's own versus others' use. arXiv:2305.01670 [cs.HC] - · arXiv
  57. [60][60] Martin Ragot, Nicolas Martin, and Salomé Cojean. 2020. AI-Generated vs. Human Artworks. A Perception Bias Towards Artificial Intelligence?. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (Chi Ea '20) . Association for Computing Machinery, New York, NY, USA, 1-10. https:

Cited by

Sorted by citations

Authors
Kloft, A. M., Welsch, R., Kosch, T., & Villa, S.
Affiliations
  • Aalto University (FI)
  • Humboldt-Universität zu Berlin (DE)
  • Ludwig-Maximilians-Universität München (DE)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
Pages
1-24
DOI
10.48550/arxiv.2309.16606
arXiv
2309.16606
OpenAlex
W4396833108
Cited by
34
References
0
Citations by year
2026: 11 · 2025: 16 · 2024: 7
FWCI
22.85
Type
conference paper
Language
en
Open access
Licence
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • Aaltodoc (Aalto University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Pain Management and Placebo Effect
Keywords
placebo effect, nocebo effect, human-AI interaction, performance expectations, information gathering, Bayesian analysis, cognitive modeling

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review