Automation Accuracy Is Good, but High Controllability May Be Better
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
When automating tasks using some form of artificial intelligence, some inaccuracy in the result is virtually unavoidable. In many cases, the user must decide whether to try the automated method again, or fix it themselves using the available user interface. We argue this decision is influenced by both perceived automation accuracy and degree of task "controllability" (how easily and to what extent an automated result can be manually modified). This relationship between accuracy and controllability is investigated in a 750-participant crowdsourced experiment using a controlled, gamified task. With high controllability, self-reported satisfaction remained constant even under very low accuracy conditions, and overall, a strong preference was observed for using manual control rather than automation, despite much slower performance and regardless of very poor controllability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
The Building Blocks of Interpretability
Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., & Mordvintsev, A. (2018) · Distill · doi:10.23915/distill.00010 · Cited by 591 · cited 3×
“On the other hand, automation communities aim at creating partners 2 that entirely carry out tasks on a user’s behalf [12] The user interface is often perceived as necessary, but secondary [15].”
BackgroundDirect manipulation vs. interface agents
Shneiderman, B., & Maes, P. (1997) · Interactions · doi:10.1145/267505.267514 · Cited by 518 · cited 2×
“The subtlety between the two visions has lead to heated debates between the two communities, an instance of which is Shneiderman and Maes’s 1997 discussion [20].”
BackgroundOh, C., Song, J., Choi, J., Kim, S., Lee, S., & Suh, B. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174223 · Cited by 301 · cited 2×
“, who measured the perceived controllability of a collaborative drawing application, and observed that people generally preferred to take initiative over the automation [14].”
Background
Cited by
Writing with AI Lowers Psychological Ownership, but Longer Prompts Can Help
Joshi, N., & Vogel, D. (2025) · Proceedings of the 7th ACM Conference on Conversational User Interfaces · doi:10.1145/3719160.3736608 · Cited by 50
Self-citationArtist Support Networks: Implications for Future Creativity Support Tools
Chung, J. J. Y., He, S., & Adar, E. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533505 · Cited by 46
Supporting Novice Researchers to Write Literature Review using Language Models
Choe, K., Park, S., Jung, S., Kim, H., Yang, J. W., Hong, H., & Seo, J. (2024) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613905.3650787 · Cited by 6
Metadata
- Authors
- Roy, Q., Zhang, F., & Vogel, D.
- Venue
- Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2019-05-02
- Pages
- 1-8
- OpenAlex
- W2941455863
- Cited by
- 134
- References
- 6
- FWCI
- 16.50
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Automation Accuracy Is Good, but High Controllability May Be Better. https://app.cochat.ai/papers/10.1145/3290605.3300750
- The paper
- Roy, Q., Zhang, F., & Vogel, D. (2019). Automation Accuracy Is Good, but High Controllability May Be Better. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300750
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.