The Placebo Effect of Artificial Intelligence in Human–Computer Interaction
Open accessThe DOI matches its Crossref record; 107 of the 107 references it cites were found in a registry or an index.
Abstract
In medicine, patients can obtain real benefits from a sham treatment. These benefits are known as the placebo effect. We report two experiments (Experiment I: N = 369; Experiment II: N = 100) demonstrating a placebo effect in adaptive interfaces. Participants were asked to solve word puzzles while being supported by no system or an adaptive AI interface. All participants experienced the same word puzzle difficulty and had no support from an AI throughout the experiments. Our results showed that the belief of receiving adaptive AI support increases expectations regarding the participant’s own task performance, sustained after interaction. These expectations were positively correlated to performance, as indicated by the number of solved word puzzles. We integrate our findings into technological acceptance theories and discuss implications for the future assessment of AI-based user interfaces and novel technologies. We argue that system descriptions can elicit placebo effects through user expectations biasing the results of user-centered studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Even non-adaptive systems are deemed adaptive and personalized if introduced as such [67].”
From The AI Ghostwriter Effect: When Users do not Perceive Ownership of AI-Generated Text but Self-Declare as Authors (2024) · cites this paper as [67]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483 · cited 1×
“In user studies, subjective measures for acceptance and usability such as the Technology Acceptance Model (TAM) [15, 16, 101], Unified Theory of Acceptance and Use of Technology (UTAUT) [46], or the NASA Task Load Index (NASA-TLX) [38] for the measurement of workload, are widely used metrics in the…”
A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies
Venkatesh, V., & Davis, F. D. (2000) · Management Science · doi:10.1287/mnsc.46.2.186.11926 · Cited by 23,066 · cited 2×
“In user studies, subjective measures for acceptance and usability such as the Technology Acceptance Model (TAM) [15, 16, 101], Unified Theory of Acceptance and Use of Technology (UTAUT) [46], or the NASA Task Load Index (NASA-TLX) [38] for the measurement of workload, are widely used metrics in the field of HCI [37].”
Venkatesh, V., Thong, J. Y. L., & Xu, X. (2012) · MIS Quarterly · doi:10.2307/41410412 · Cited by 15,756 · cited 9×
“Technology acceptance models such as the UTAUT [104] emphasize the role of subjective evaluation, as participants’ demographics, prior experience, and so on can mitigate both performance expectations and their influence over user behavior.”
InfluentialDevelopment of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003 · cited 8×
“After each adaptivity, participants filled in a NASA-TLX questionnaire.”
InfluentialConfirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267 · cited 2×
“One could explain this continuance of belief in terms of confirmation bias [76]), which describes the propensity of participants to gather evidence for confirming a hypothesis.”
(1985) · Cited by 6,185
Ekman, P., & Friesen, W. V. (1975) · Cited by 4,604
NASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297 · cited 3×
“Following that, they were asked to solve the 20 word puzzles and to fill in a NASA-TLX questionnaire [37, 38].”
The aligned rank transform for nonparametric factorial analyses using only anova procedures
Wobbrock, J. O., Findlater, L., Gergle, D., & Higgins, J. J. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978963 · Cited by 2,854 · cited 1×
“We will test user expectations before and after interaction with the system in a rank-aligned repeated-measures Analysis of Variance (ANOVA) [49].”
Beecher, H. K. (1955) · Journal of the American Medical Association · doi:10.1001/jama.1955.02960340022006 · Cited by 1,842 · cited 3×
“Such a placebo can alleviate pain [60] or support the treatment of ailments [3, 48], thereby providing an effective medical treatment without an illness-specific mechanism of effect.”
Research works printed but not linked: 5
- T. Bentley. 2000. Biasing web site user evaluations: A study. In Proceedings of the Conference of the Computer Human Interaction Special Interest Group of the Ergonomics Society of Australia. 130–134.
- Aviarini Indrati, Edi Minaji, Sugiharti Binastuti, and Philipus Dwi Raharjo. 2014. Comparation of model unified theory of acceptance and use technology (UTAUT) And technology acceptance model (TAM) for internet adoption of credit union staff. In Proceedings of the 1st International Credit Union Conference on Social Micro.
- Irving Kirsch. 1997. Specifying nonspecifics: Psychological mechanisms of placebo effects. In The Placebo Effect: An Interdisciplinary Exploration Anne Harrington (Ed.). Harvard University Press 166–186.
- 10.2307/41410412 · DOI · Paper page
- 10.1111/j.1540-5915.2010.00292.x · DOI · Paper page
Cited by
Items 101–103 of 103
Huang, T., Kong, W., Liang, J., Ding, Z., Li, H., & Zhang, X. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8429-9_6
Analysis of Piano Playing Techniques Research Using Long Short-Term Memory Networks (LSTM)
Su, T. (2024) · International Conference on Machine Intelligence and Digital Applications (MIDA) · doi:10.1145/3662739.3672177
“The study of piano performance techniques [1, 2] has always been an important topic in the fields of musicology and human-computer interaction.”
Performance and Metacognition Disconnect when Reasoning in Human-AI Interaction
Fernandes, D., Villa, S., Nicholls, S., Haavisto, O., Buschek, D., Schmidt, A., Kosch, T., Shen, C., & Welsch, R. (2024) · arXiv · doi:10.48550/arxiv.2409.16708 · Cited by 71
Self-citation
Metadata
- Authors
- Kosch, T., Welsch, R., Chuang, L., & Schmidt, A.
- Affiliations
- Utrecht University (NL)
- Ludwig-Maximilians-Universität München (DE)
- Chemnitz University of Technology (DE)
- Venue
- ACM Transactions on Computer-Human Interaction
- Published
- 2022-12-31
- Volume
- 29
- Issue
- 6
- Pages
- 1-32
- ISSN
- 1073-0516 (ISSN-L)
- 1557-7325
- OpenAlex
- W4224105048
- Cited by
- 137
- References
- 106
- Citations by year
- 2026: 33 · 2025: 38 · 2024: 30 · 2023: 29 · 2022: 7
- FWCI
- 12.85
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Transactions on Computer-Human Interaction (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Instant Teaming between Humans and Production Systems
- Saxony State Ministry of Science and Art (SMWK3-7304/35/3-2021/4819)
- European Union’s Horizon 2020 Programme (683008 AMPLIFY)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- placebo effect, artificial intelligence, adaptive user interfaces, user expectations, task performance, technology acceptance, system description, user-centered evaluation, word puzzles
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.