CoChat
Sign Up
PreprintCited by 31Innovative Human-Technology InteractionEthics and Social Impacts of AIHuman-Automation Interaction and Safety

Dice in the Black Box: User Experiences with an Inscrutable Algorithm

Open access
Springer, Aaron, Hollis, Victoria, Whittaker, Steve·2018·arXiv
DOI 10.48550/arxiv.1812.03219arXiv 1812.03219
Open this paper in CoChat — free accountOpen PDF

Abstract

We demonstrate that users may be prone to place an inordinate amount of trust in black box algorithms that are framed as intelligent. We deploy an algorithm that purportedly assesses the positivity and negativity of a users' writing emotional writing. In actuality, the algorithm responds in a random fashion. We qualitatively examine the paths to trust that users followed while testing the system. In light of the ease with which users may trust systems exhibiting "intelligent behavior" we recommend corrective approaches.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1812.03219

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In prior experiments, users judged the E-meter algorithm to be accurate but not perfect; the median response for accuracy was 'Accurate' (6 on a 7-point Likert scale).

    “In prior experiments [78,80], users judged this algorithm to be accurate but not perfect; the median response for accuracy was “Accurate” (“6” on a 7-point Likert scale with the highest rating (“7”) being “Very Accurate”).”

    From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [78]

    The Results (p. 2) report a median E-meter accuracy rating of 5, labelled “Slightly Accurate,” on a 7-point scale—not 6, labelled “Accurate.” The paper cautions that its sample of 12 is too small for a representative quantitative estimate. (LLM, conf=0.94)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • The E-meter system and protocol have been successfully deployed in multiple previous experiments.

    “The E-meter system and protocol have been successfully deployed in multiple previous experiments [78,80].”

    From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [78]

    The E-meter System and Results sections (p. 2) describe the system and its use with 12 participants, supporting deployment in an experiment. The Discussion (p. 3) calls this a small pilot; the paper does not establish that the E-meter system and protocol were deployed in multiple previous experiments. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Algorithm accuracy evaluations closely relate to expectation violation.

    “First, our results replicate and extend other studies showing that algorithm accuracy evaluations closely relate to expectation violation [43,78,80].”

    From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [78]

    The E-meter System section (p. 2) says the authors expected the random feedback to violate some users’ expectations, and the Results (p. 3) connect mismatches with users’ accuracy judgments. But the Results also say participants often disregarded contradictory feedback and still rated the system slightly accurate. The paper does not establish the claimed close relationship between expectation violation and accuracy evaluations. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • The E-meter system and protocol have been successfully deployed in multiple previous experiments.

    “The E-meter system and protocol have been successfully deployed in multiple previous experiments [76,78].”

    From Progressive disclosure (2019) · cites this paper as [76]

    The E-meter System and Materials and Methodology sections (p. 2) describe the system and procedure used with participants, supporting this paper as an example of a prior deployment. The Discussion (p. 3) calls it a small pilot; this source does not establish that the E-meter system and protocol were deployed in multiple previous experiments. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 3

  1. Gilbert, D. T., Pinel, E. C., Wilson, T. D., Blumberg, S. J., & Wheatley, T. P. (1998). Immune neglect: a source of durability bias in affective forecasting. Journal of Personality and Social Psychology , 75 (3), 617.
  2. Hollis, V., Konrad, A., Springer, A., Antoun, C., Antoun, M., Martin, R., & Whittaker, S. (2016). What Does All This Data Mean For My Future Mood? Actionable Analytics & Targeted Reflection for Emotional Well-Being. Under review.
  3. Petty, R. E., Cacioppo, J. T., & Goldman, R. (1981). Personal involvement as a determinant of argument-based persuasion. Journal of Personality and Social Psychology , 41 (5), 847.

Cited by

Sorted by citations

Related

Authors
Springer, A., Hollis, V., & Whittaker, S.
Affiliations
University of California, Santa Cruz (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2018
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1812.03219
arXiv
1812.03219
OpenAlex
W2902302393
Cited by
29
References
0
Citations by year
2024: 2 · 2023: 1 · 2022: 5 · 2021: 3 · 2020: 3 · 2019: 5 · 2018: 8 · 2017: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • National Conference on Artificial Intelligence
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
Keywords
trust in algorithms, black-box algorithms, algorithmic opacity, perceived intelligence, user experience, random response, algorithmic bias, trustworthiness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review