The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface
Abstract
The rising prevalence of algorithmic interfaces, such as curated feeds in online news, raises new questions for designers, scholars, and critics of media. This work focuses on how transparent design of algorithmic interfaces can promote awareness and foster trust. A two-stage process of how transparency affects trust was hypothesized drawing on theories of information processing and procedural justice. In an online field experiment, three levels of system transparency were tested in the high-stakes context of peer assessment. Individuals whose expectations were violated (by receiving a lower grade than expected) trusted the system less, unless the grading algorithm was made more transparent through explanation. However, providing too much information eroded this trust. Attitudes of individuals whose expectations were met did not vary with transparency. Results are discussed in terms of a dual process model of attitude change and the depth of justification of perceived inconsistency. Designing for trust requires balanced interface transparency - not too little and not too much.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It should be noted that most of these studies that tested the effect of algorithmic transparency on trust employed samples with limited resemblance to the overall population (e.g. Kizilcec 2016) or are about decisions that have a little direct impact on citizen lives (e.g. Schmidt et al. 2020).”
From Explaining Why the Computer Says No: Algorithmic Transparency Affects the Perceived Trustworthiness of Automated Decision‐Making (2023) · cites this paper as Kizilcec 2016
In the citing paper
“There is some empirical evidence showing that algorithmic transparency can indeed increase user trust in its decisions.”
From Explaining Why the Computer Says No: Algorithmic Transparency Affects the Perceived Trustworthiness of Automated Decision‐Making (2023) · cites this paper as Kizilcec (2016)
In the citing paper
“At the same time, this study also found that too much detailed information backfired and led to less perceived trustworthiness.”
From Explaining Why the Computer Says No: Algorithmic Transparency Affects the Perceived Trustworthiness of Automated Decision‐Making (2023) · cites this paper as Kizilcec (2016)
In the citing paper
“Kizilcec (2016) demonstrated that short textual explanations build subjective trust in an algorithm's decision.”
From Exploring the effects of human-centered AI explanations on trust and reliance (2023) · cites this paper as Kizilcec, 2016
In the citing paper
“Some research suggests that the way we present transparency may account for these contradictory results [28,43].”
From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [43]
13 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
The Elaboration Likelihood Model of Persuasion
Petty, R. E., & Cacioppo, J. T. (1986) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60214-2 · Cited by 8,822 · cited 3×
“less likely to examine information thoroughly and more likely to rely on general impressions or their own mood [18].”
BackgroundThe Social Psychology of Procedural Justice
Lind, E. A., & Tyler, T. R. (1988) · Critical Issues in Social Justice · doi:10.1007/978-1-4899-2115-4 · Cited by 6,585 · cited 2×
“Consistent with procedural justice theory [15], a procedural explanation re-instilled trust in those whose expectations were violated.”
Background(1997) · Choice Reviews Online · doi:10.5860/choice.34-3702 · Cited by 4,887
The Elaboration Likelihood Model of Persuasion
Petty, R. E., & Cacioppo, J. T. (1986) · Communication and Persuasion · doi:10.1007/978-1-4612-4964-1_1 · Cited by 3,202
Explaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731
Manjunath, B. S., Ohm, J.-R., Vasudevan, V. V., & Yamada, A. (2001) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/76.927424 · Cited by 1,666
On-line trust: concepts, evolving themes, a model
Corritore, C. L., Kracher, B., & Wiedenbeck, S. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00041-7 · Cited by 1,200
Nonverbal expectancy violations: Model elaboration and application to immediacy behaviors
Burgoon, J. K., & Hale, J. L. (1988) · Communication Monographs · doi:10.1080/03637758809376158 · Cited by 828
Proceedings of the 7th International Conference on Educational Data Mining.
Stamper, J. C., Pardos, Z. A., Mavrikis, M., & McLaren, B. M. (2014) · Educational Data Mining · Cited by 667
Eslami, M., Rickman, A., Vaccaro, K., Aleyasen, A., Vuong, A., Karahalios, K., Hamilton, K., & Sandvig, C. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702556 · Cited by 627
The Impact of Self- and Peer-Grading on Student Learning
Sadler, P., & Good, E. (2006) · Educational Assessment · doi:10.1207/s15326977ea1101_1 · Cited by 535 · cited 2×
“Peer grading is often supplemented by self grading to encourage the development of self-evaluative skills and is generally thought to “augment student learning” [25].”
BackgroundThe mechanics of trust: A framework for research and design
Riegelsberger, J., Sasse, M. A., & McCarthy, J. D. (2005) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2005.01.001 · Cited by 508 · cited 1×
“Trust is a key concern in the design of technology, as it affects the initial adoption and continued use of technologies [4, 24].”
BackgroundRecommendation Agents for Electronic Commerce: Effects of Explanation Facilities on Trusting Beliefs
Wang, W., & Benbasat, I. (2007) · Journal of Management Information Systems · doi:10.2753/mis0742-1222230410 · Cited by 498
The MPEG-7 visual standard for content description-an overview
Sikora, T. (2001) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/76.927422 · Cited by 496
The role of transparency in recommender systems
Sinha, R., & Swearingen, K. (2002) · CHI '02 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/506443.506619 · Cited by 477
The effects of transparency on trust in and acceptance of a content-based art recommender
Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-008-9051-3 · Cited by 448
Accepting threatening information: Self‐affirmation and the reduction of defensive biases
Sherman, D. K., & Cohen, G. L. (2002) · Current Directions in Psychological Science · doi:10.1111/1467-8721.00182 · Cited by 435
Peer and self assessment in massive online classes
Kulkarni, C., Wei, K. P., Le, H., Chia, D., Papadopoulos, K., Cheng, J., Koller, D., & Klemmer, S. R. (2013) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2505057 · Cited by 390 · cited 3×
“Peer assessment is a proven method for scaling the grading of a large number of assessments, such as is required in massive open online courses (MOOCs) [14, 19].”
BackgroundMethodUnderstanding User Beliefs About Algorithmic Curation in the Facebook News Feed
Rader, E., & Gray, R. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702174 · Cited by 386 · cited 1×
“Yet many users are unaware of this algorithm [7] or develop personal beliefs about it [22].”
BackgroundThe Trouble with Transparency: A Critical Review of Openness in e‐Government
Bannister, F., & Connolly, R. (2011) · Policy & Internet · doi:10.2202/1944-2866.1076 · Cited by 368
Students’ perceptions about peer assessment for writing: their origin and impact on revision work
Kaufman, J. H., & Schunn, C. D. (2011) · Instructional Science · doi:10.1007/s11251-010-9133-6 · Cited by 335
A case for interaction: a study of interactive information retrieval behavior and effectiveness
Koenemann, J., & Belkin, N. J. (1996) · Proceedings of the SIGCHI conference on Human factors in computing systems common ground - CHI '96 · doi:10.1145/238386.238487 · Cited by 327
Sambell, K., McDowell, L., & Brown, S. (1997) · Studies in Educational Evaluation · doi:10.1016/s0191-491x(97)86215-3 · Cited by 320
University Students’ Perceptions of and Attitudes Toward (Online) Peer Assessment
Wen, M. L., & Tsai, C.-C. (2006) · Higher Education · doi:10.1007/s10734-004-6375-8 · Cited by 273
Tuned Models of Peer Assessment in MOOCs
Piech, C., Huang, J., Chen, Z., Do, C., Ng, A., & Koller, D. (2013) · arXiv · doi:10.48550/arxiv.1307.2579 · Cited by 262 · cited 3×
“Piech and colleagues [19] were able to improve accuracy by 30% using algorithms that adjust grades for grader bias and reliability.”
BackgroundMethodNew Assessment Forms in Problem-based Learning: The value-added of the students' perspective
Segers, M., & Dochy, F. (2001) · Studies in Higher Education · doi:10.1080/03075070120076291 · Cited by 253
Trust-inspiring explanation interfaces for recommender systems
Pu, P., & Chen, L. (2007) · Knowledge-Based Systems · doi:10.1016/j.knosys.2007.04.004 · Cited by 248
Toward establishing trust in adaptive agents
Glass, A., McGuinness, D. L., & Wolverton, M. (2008) · Proceedings of the 13th international conference on Intelligent user interfaces · doi:10.1145/1378773.1378804 · Cited by 211
A path to understanding the effects of algorithm awareness
Hamilton, K., Karahalios, K. G., Sandvig, C., & Eslami, M. (2014) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2559206.2578883 · Cited by 156
Explanation and trust: what to tell the user in security and AI?
Pieters, W. (2011) · Ethics and Information Technology · doi:10.1007/s10676-010-9253-3 · Cited by 153
(2020) · Springer eBooks · doi:10.1007/978-3-319-15347-6_301389 · Cited by 136
Transparent Queries: investigation users' mental models of search engines
Muramatsu, J., & Pratt, W. (2001) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/383952.383991 · Cited by 101
Explanation facilities and interactive systems
Johnson, H., & Johnson, P. (1993) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/169891.169951 · Cited by 78
eTrust: Forming Relationships in the Online World
Sztompka, P. (2010) · Contemporary Sociology A Journal of Reviews · doi:10.1177/0094306110380384m · Cited by 44
The Future of Data-Enriched Assessment.
Thille, C., Schneider, E., Kizilcec, R. F., Piech, C., Halawa, S. A., & Greene, D. K. (2014) · Research & Practice in Assessment · Cited by 33
Self-citationAn Overview of MPEG-7 Motion Descriptors and Their Applications
Divakaran, A. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44692-3_5 · Cited by 13
Research works printed but not linked: 2
- Buchanan Bruce G, 1984, Edward Hance Shortliffe, and others
- Payne Stephen J, 2003, Users' mental models: the very ideas. HCI models, theories, and frameworks: Toward a multidisciplinary science
Cited by
The future of employment: How susceptible are jobs to computerisation?
Frey, C. B., & Osborne, M. A. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.08.019 · Cited by 8,727
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921
Shin, D. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102551 · Cited by 1,548
Tsai, C.-W., Lai, C.-F., Chao, H.-C., & Vasilakos, A. V. (2015) · Journal of Big Data · doi:10.1186/s40537-015-0030-3 · Cited by 818
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691
“A notable exception is Kizilcec, who studied the effect of levels of transparency on perceptions of procedural justice [35].”
Background
Metadata
- Authors
- Kizilcec, R. F.
- Affiliations
- Stanford University (US)
- Corresponding author
- Kizilcec, René F.
- ORCID
- Kizilcec, René F.
- Venue
- Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2016-05-07
- Pages
- 2390-2395
- OpenAlex
- W2403343111
- Cited by
- 615
- References
- 36
- Citations by year
- 2026: 127 · 2025: 83 · 2024: 62 · 2023: 49 · 2022: 54 · 2021: 54 · 2020: 64 · 2019: 40 · 2018: 23 · 2017: 10 · 2016: 22
- FWCI
- 17.06
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- system transparency, trust in algorithms, information overload, peer assessment, automated grading, procedural justice, information processing theory, dual-process models, attitude change
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.