For What It’s Worth: Humans Overwrite Their Economic Self-interest to Avoid Bargaining With AI Systems
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
As algorithms are increasingly augmenting and substituting human decision-making, understanding how the introduction of computational agents changes the fundamentals of human behavior becomes vital. This pertains to not only users, but also those parties who face the consequences of an algorithmic decision. In a controlled experiment with 480 participants, we exploit an extended version of two-player ultimatum bargaining where responders choose to bargain with either another human, another human with an AI decision aid or an autonomous AI-system acting on behalf of a passive human proposer. Our results show strong responder preferences against the algorithm, as most responders opt for a human opponent and demand higher compensation to reach a contract with autonomous agents. To map these preferences to economic expectations, we elicit incentivized subject beliefs about their opponent’s behavior. The majority of responders maximize their expected value when this is line with approaching the human proposer. In contrast, responders predicting income maximization for the autonomous AI-system overwhelmingly override economic self-interest to avoid the algorithm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Machines and Mindlessness: Social Responses to Computers
Nass, C., & Moon, Y. (2000) · Journal of Social Issues · doi:10.1111/0022-4537.00153 · Cited by 4,022 · cited 1×
“Building on early experimental work within the computers are social actors (CASA) paradigm [123], a lot of research has shown that humans tend to treat machine actors as social agents by e.”
BackgroundWhat is AI Literacy? Competencies and Design Considerations
Long, D., & Magerko, B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376727 · Cited by 3,241 · cited 1×
“This ensured a straightforward decision environment without confounds relating to the effects of technical details, degrees of interpretability or cognitive overload [81, 115].”
BackgroundFairness and retaliation: the economics of reciprocity
Fehr, E., & Gächter, S. (2000) · Journal of Economic Perspectives · doi:10.1257/jep.14.3.159 · Cited by 3,239
Algorithms at Work: The New Contested Terrain of Control
Kellogg, K. C., Valentine, M. A., & Christin, A. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0174 · Cited by 2,732 · cited 1×
“In organizations, AI-technologies can be leveraged to monitor, rate and reward employees [95], determine who gets hired [100, 109], and must function within interdisciplinary team environments that involve everyday negotiations such as allocating workloads [10, 70].”
BackgroundThird-party punishment and social norms
Fehr, E., & Fischbacher, U. (2004) · Evolution and Human Behavior · doi:10.1016/s1090-5138(04)00005-4 · Cited by 2,417
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Resistance to Medical Artificial Intelligence
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019) · Journal of Consumer Research · doi:10.1093/jcr/ucz013 · Cited by 2,015 · cited 1×
“Similar behavioral patterns have been observed for delegation in moral domains [63], subjective tasks [33], morally controversial investments [125], moral dilemmas and autonomous driving [48, 162], managerial [106] or medical decision-making [116].”
BackgroundTask-Dependent Algorithm Aversion
Castelo, N., Bos, M. W., & Lehmann, D. R. (2019) · Journal of Marketing Research · doi:10.1177/0022243719851788 · Cited by 1,500 · cited 2×
“Yet, when tasks are more objective [33, 106], humans lack expertise [114], humans retain some agency [47], humans and machines share some common rationale [144] or after prior exposure [99], people might also prefer or at least accept machine decision making.”
BackgroundThe price is unfair! A conceptual framework of price fairness perceptions
Xia, L., Monroe, K. B., & Cox, J. L. (2004) · Journal of Marketing · doi:10.1509/jmkg.68.4.1.42733 · Cited by 1,474 · cited 2×
“algorithmic pricing [22, 111, 159] relate to their purchasing choices, a firm’s competitiveness, profits, overall economic efficiency, and consumer welfare.”
BackgroundValidating vignette and conjoint survey experiments against real-world behavior
Hainmueller, J., Hangartner, D., & Yamamoto, T. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1416587112 · Cited by 1,344 · cited 1×
“…(too much) between the research environment and the outside world [108], there is a vast literature documenting the predictive power of abstract experimental findings for real-world decisions [14, 30, 35, 60, 71, 78, 92, 104, 156], although there can be experimentally unobserved confounds [62].”
BackgroundLee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338 · cited 2×
“Yet, when tasks are more objective [33, 106], humans lack expertise [114], humans retain some agency [47], humans and machines share some common rationale [144] or after prior exposure [99], people might also prefer or at least accept machine decision making.”
BackgroundDietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230
Social Image and the 50-50 Norm: A Theoretical and Experimental Analysis of Audience Effects
(2009) · Econometrica · doi:10.3982/ecta7384 · Cited by 1,081 · cited 1×
“broadly agree on context-dependent distributional norms like the 50/50-rule, reciprocate actions that signal trust, and feel obligated to fulfill responsibilities within teams [5, 6, 53].”
BackgroundIn AI we trust? Perceptions about automated decision-making by artificial intelligence
Araujo, T., Helberger, N., Kruikemeier, S., & de Vreese, C. H. (2020) · AI & SOCIETY · doi:10.1007/s00146-019-00931-w · Cited by 1,038
Conditional Cooperation and Voluntary Contributions to Public Goods
Keser, C., & Van Winden, F. (2000) · The Scandinavian Journal of Economics · doi:10.1111/1467-9442.00182 · Cited by 800
People are averse to machines making moral decisions.
Bigman, Y. E., & Gray, K. (2018) · Cognition · doi:10.1016/j.cognition.2018.08.003 · Cited by 738 · cited 1×
“Bigman and Gray (2018) document that human aversions towards moral decisions by machines are consistent across a variety of social domains and in part caused by a perceived lack of mind [18].”
BackgroundArtificial Intelligence, Algorithmic Pricing, and Collusion
Calvano, E., Calzolari, G., Denicolò, V., & Pastorello, S. (2020) · American Economic Review · doi:10.1257/aer.20190623 · Cited by 708
Tay, B., Jung, Y., & Park, T. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.05.014 · Cited by 490 · cited 1×
“ascribing stereotypes [12, 87, 147], reacting to social cues [3, 44, 45], being triggered into social conformity [79, 139] or acting reciprocal [59, 93].”
BackgroundMeasuring price discrimination and steering on e-commerce web sites
Hannak, A., Soeller, G., Lazer, D., Mislove, A., & Wilson, C. (2014) · Proceedings of the 2014 Conference on Internet Measurement Conference · doi:10.1145/2663716.2663744 · Cited by 405 · cited 1×
“In consumer markets, algorithms determine the allocation of scarce goods through differential pricing [27, 29, 36, 37, 72, 73], automate financial transactions like credit-score loaning [107, 138, 151, 164] or trading [19, 32, 43, 74, 110], augment decision processes in credence goods markets [96], and are primed to a…”
BackgroundTournament Theory: Thirty Years of Contests and Competitions
Connelly, B. L., Tihanyi, L., Crook, T. R., & Gangloff, K. A. (2014) · Journal of Management · doi:10.1177/0149206313498902 · Cited by 338
Bias in Online Freelance Marketplaces: Evidence from TaskRabbit and Fiverr
Hannák, A., Wagner, C., Garcia, D., Mislove, A., Strohmaier, M., & Wilson, C. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998327 · Cited by 286 · cited 1×
“In consumer markets, algorithms determine the allocation of scarce goods through differential pricing [27, 29, 36, 37, 72, 73], automate financial transactions like credit-score loaning [107, 138, 151, 164] or trading [19, 32, 43, 74, 110], augment decision processes in credence goods markets [96], and are primed to a…”
BackgroundUpdates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Bansal, G., Nushi, B., Kamar, E., Weld, D. S., Lasecki, W. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012429 · Cited by 284 · cited 1×
“In organizations, AI-technologies can be leveraged to monitor, rate and reward employees [95], determine who gets hired [100, 109], and must function within interdisciplinary team environments that involve everyday negotiations such as allocating workloads [10, 70].”
BackgroundHuman-machine collaboration in managerial decision making
Haesevoets, T., De Cremer, D., Dierckx, K., & Van Hiel, A. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106730 · Cited by 280 · cited 1×
“In organizations, AI-technologies can be leveraged to monitor, rate and reward employees [95], determine who gets hired [100, 109], and must function within interdisciplinary team environments that involve everyday negotiations such as allocating workloads [10, 70].”
BackgroundA Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
De-Arteaga, M., Fogliato, R., & Chouldechova, A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376638 · Cited by 221 · cited 1×
“Consequently, there has been a lot of behavioral research focusing on how humans make use of or rely algorithmic and AI-systems [42, 46, 114], or cooperate with them [41, 66, 117, 149, 168].”
BackgroundHighly automated job interviews: Acceptance under the influence of stakes
Langer, M., König, C. J., & Papathanasiou, M. (2019) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12246 · Cited by 203 · cited 1×
“in evaluating job performance or hiring decisions can directly influence the system’s effectiveness [20, 94, 102, 158].”
BackgroundMapping the Ethicality of Algorithmic Pricing: A Review of Dynamic and Personalized Pricing
Seele, P., Dierksmeier, C., Hofstetter, R., & Schultz, M. D. (2021) · Journal of Business Ethics · doi:10.1007/s10551-019-04371-w · Cited by 196
Behavioural evidence for a transparency–efficiency tradeoff in human–machine cooperation
Ishowo-Oloko, F., Bonnefon, J.-F., Soroye, Z., Crandall, J., Rahwan, I., & Rahwan, T. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0113-5 · Cited by 178
Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
Lu, Z., & Yin, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445562 · Cited by 131 · cited 1×
“Consequently, there has been a lot of behavioral research focusing on how humans make use of or rely algorithmic and AI-systems [42, 46, 114], or cooperate with them [41, 66, 117, 149, 168].”
BackgroundWhen Do People Want AI to Make Decisions?
Kramer, M. F., Schaich Borg, J., Conitzer, V., & Sinnott-Armstrong, W. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278752 · Cited by 83 · cited 1×
“Yet, when tasks are more objective [33, 106], humans lack expertise [114], humans retain some agency [47], humans and machines share some common rationale [144] or after prior exposure [99], people might also prefer or at least accept machine decision making.”
BackgroundMarch, C. (2021) · Journal of Economic Psychology · doi:10.1016/j.joep.2021.102426 · Cited by 72
Second Chance for a First Impression? Trust Development in Intelligent System Interaction
Tolmeijer, S., Gadiraju, U., Ghantasala, R., Gupta, A., & Bernstein, A. (2021) · Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3450613.3456817 · Cited by 67 · cited 1×
“Consequently, there has been a lot of behavioral research focusing on how humans make use of or rely algorithmic and AI-systems [42, 46, 114], or cooperate with them [41, 66, 117, 149, 168].”
BackgroundSelf-citation
Cited by
Zhang, R., Duan, W., Flathmann, C., McNeese, N., Freeman, G., & Williams, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610072 · Cited by 134
Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems
He, G., Kuiper, L., & Gadiraju, U. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581025 · Cited by 107
Self-citationA new sociology of humans and machines
Tsvetkova, M., Yasseri, T., Pescetelli, N., & Werner, T. (2024) · arXiv · doi:10.48550/arxiv.2402.14410 · Cited by 83
Effect of Explanation Conceptualisations on Trust in AI-assisted Credibility Assessment
Pareek, S., van Berkel, N., Velloso, E., & Goncalves, J. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686922 · Cited by 42
Salimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41
“For example, in high-stake scenarios such as in the medical [39, 61, 67, 97], legal [6, 81, 86, 131], and financial [27, 36, 43, 45, 46] domains, individuals tend to exhibit a preference for human decision-makers over AI systems.”
BackgroundSelf-citation
Metadata
- Authors
- Erlei, A., Das, R., Meub, L., Anand, A., & Gadiraju, U.
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-27
- Pages
- 1-18
- OpenAlex
- W4225006930
- Cited by
- 33
- References
- 31
- FWCI
- 7.70
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.