Communicating Algorithmic Process in Online Behavioral Advertising
The DOI matches its Crossref record; 43 of the 44 references it cites were found in a registry or an index.
Abstract
Advertisers develop algorithms to select the most relevant advertisements for users. However, the opacity of these algorithms, along with their potential for violating user privacy, has decreased user trust and preference in behavioral advertising. To mitigate this, advertisers have started to communicate algorithmic processes in behavioral advertising. However, how revealing parts of the algorithmic process affects users' perceptions towards ads and platforms is still an open question. To investigate this, we exposed 32 users to why an ad is shown to them, what advertising algorithms infer about them, and how advertisers use this information. Users preferred interpretable, non-creepy explanations about why an ad is presented, along with a recognizable link to their identity. We further found that exposing users to their algorithmically-derived attributes led to algorithm disillusionment---users found that advertising algorithms they thought were perfect were far from it. We propose design implications to effectively communicate information about advertising algorithms.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Privacy in Context: Technology, Policy, and the Integrity of Social Life
Nissenbaum, H. (2009) · doi:10.1515/9780804772891 · Cited by 2,129 · cited 1×
“Most algorithmic decision-making systems do not communicate their inner workings to users [39], resulting in information asymmetry—a disparity in what is visible to different parties to a system [34].”
BackgroundExplaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731 · cited 1×
“Recommender systems incorporating explanations produced more user acceptance of, confidence in and trust in recommendations than those without explanations [47, 21, 40, 54].”
BackgroundInvestigating Ad Transparency Mechanisms in Social Media: A Case Study of Facebook's Explanations
Andreou, A., Venkatadri, G., Goga, O., Gummadi, K. P., Loiseau, P., & Mislove, A. (2018) · Proceedings 2018 Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23191 · Cited by 1,477
The Black Box Society: The Secret Algorithms That Control Money and Information
(2016) · Contemporary Sociology: A Journal of Reviews · doi:10.1177/0094306116641409c · Cited by 1,334
A New Algorithmic Identity: Soft Biopolitics and the Modulation of Control
Cheney-Lippold, J. (2011) · Theory, Culture & Society · doi:10.1177/0263276411424420 · Cited by 684
Eslami, M., Rickman, A., Vaccaro, K., Aleyasen, A., Vuong, A., Karahalios, K., Hamilton, K., & Sandvig, C. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702556 · Cited by 627
Self-citationHow Much Information?: Effects of Transparency on Trust in an Algorithmic Interface
Kizilcec, R. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858402 · Cited by 615
Media Technologies: Essays on Communication, Materiality, and Society
(2014) · The MIT Press eBooks · doi:10.7551/mitpress/9780262525374.001.0001 · Cited by 580
Algorithmic Transparency in the News Media
Diakopoulos, N., & Koliska, M. (2017) · Digital Journalism · doi:10.1080/21670811.2016.1208053 · Cited by 515 · cited 1×
“Increased process communication has also been recommended for social matching systems [28], team formation tools [26], rating platforms [15], and algorithmic journalism [12] to build informed and trustworthy interactions between users and systems; and yet effectively revealing an algorithmic process is a serious chall…”
BackgroundRecommendation Agents for Electronic Commerce: Effects of Explanation Facilities on Trusting Beliefs
Wang, W., & Benbasat, I. (2007) · Journal of Management Information Systems · doi:10.2753/mis0742-1222230410 · Cited by 498
The role of transparency in recommender systems
Sinha, R., & Swearingen, K. (2002) · CHI '02 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/506443.506619 · Cited by 477
Privacy in Context: Technology, Policy, and the Integrity of Social Life
Han, S. (2011) · Journal of Information Policy · doi:10.5325/jinfopoli.1.2011.0149 · Cited by 462
Too much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388 · cited 1×
“This finding recalls previous work that showed an oversimplified explanation of how intelligent agents work can cause users to lose trust in the system [30].”
ResultFirst I "like" it, then I hide it: Folk Theories of Social Feeds
Eslami, M., Karahalios, K., Sandvig, C., Vaccaro, K., Rickman, A., Hamilton, K., & Kirlik, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858494 · Cited by 375
Self-citationSmart, useful, scary, creepy: perceptions of online behavioral advertising
Ur, B., Leon, P. G., Cranor, L. F., Shay, R., & Wang, Y. (2012) · Proceedings of the Eighth Symposium on Usable Privacy and Security · doi:10.1145/2335356.2335362 · Cited by 313
Trust-inspiring explanation interfaces for recommender systems
Pu, P., & Chen, L. (2007) · Knowledge-Based Systems · doi:10.1016/j.knosys.2007.04.004 · Cited by 248
The limits of transparency: Data brokers and commodification
Crain, M. (2018) · New Media & Society · doi:10.1177/1461444816657096 · Cited by 237
Algorithmic Accountability Reporting: On the Investigation of Black Boxes
Diakopoulos, N. A. (2014) · Columbia Academic Commons (Columbia University) · doi:10.7916/d8zk5tw2 · Cited by 221
The Aisles Have Eyes: How Retailers Track Your Shopping, Strip Your Privacy, and Define Your Power
Turow, J. (2017) · Cited by 167
A Taxonomy for Generating Explanations in Recommender Systems
Friedrich, G., & Zanker, M. (2011) · AI Magazine · doi:10.1609/aimag.v32i3.2365 · Cited by 149
EU regulations on algorithmic decision-making and a "right to explanation".Preprint
Goodman, B., & Flaxman, S. (2016) · arXiv (Cornell University) · Cited by 134
The Intention Economy: When Customers Take Charge
Searls, D. (2012) · Cited by 122
Tene, O., & Polonetsky, J. (2011) · Minnesota journal of law science & technology · doi:10.24926/15529541.3472 · Cited by 109
Eslami, M., Vaccaro, K., Karahalios, K., & Hamilton, K. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14898 · Cited by 90
Self-citationAwareness of Behavioral Tracking and Information Privacy Concern in Facebook and Google
Rader, E. J. (2014) · Symposium On Usable Privacy and Security · Cited by 85
Oh, C., Lee, T., Kim, Y., Park, S., Kwon, S., & Suh, B. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025539 · Cited by 84
Internet Co-Regulation: European Law, Regulatory Governance and Legitimacy in Cyberspace
Marsden, C. T. (2011) · Cited by 79
Jahanbakhsh, F., Fu, W., Karahalios, K. G., Marinov, D., & Bailey, B. P. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3026011 · Cited by 75
Self-citationFolk Models of Online Behavioral Advertising
Yao, Y., Lo Re, D., & Wang, Y. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998316 · Cited by 69
AdReveal: improving transparency into online targeted advertising
Liu, B., Sheth, A., Weinsberg, U., Chandrashekar, J., & Govindan, R. (2013) · Proceedings of the Twelfth ACM Workshop on Hot Topics in Networks · doi:10.1145/2535771.2535783 · Cited by 68
The intention economy: when customers take charge
(2012) · Choice Reviews Online · doi:10.5860/choice.50-2168 · Cited by 68
A personalized approach to web privacy: awareness, attitudes and actions
Wills, C. E., & Zeljkovic, M. (2011) · Information Management & Computer Security · doi:10.1108/09685221111115863 · Cited by 66
What do online behavioral advertising privacy disclosures communicate to users?
Leon, P. G., Cranshaw, J., Cranor, L. F., Graves, J., Hastak, M., Ur, B., & Xu, G. (2012) · Proceedings of the 2012 ACM workshop on Privacy in the electronic society · doi:10.1145/2381966.2381970 · Cited by 62
Usable Transparency with the Data Track
Angulo, J., Fischer‐Hübner, S., Pulls, T., & Wästlund, E. (2015) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2702613.2732701 · Cited by 60
Friedman, L. M. (2017) · Yale University Press eBooks · doi:10.12987/9780300147209 · Cited by 57
Watching Them Watching Me: Browser Extensions Impact on User Privacy Awareness and Concern
Schaub, F., Marella, A., Kalvani, P., Ur, B., Pan, C., Forney, E., & Cranor, L. F. (2016) · Workshop on Usable Security (USEC) · doi:10.14722/usec.2016.23017 · Cited by 56
MyAdChoices: Bringing Transparency and Control to Online Advertising
Parra-Arnau, J., Achara, J. P., & Castelluccia, C. (2017) · ACM Transactions on the Web · doi:10.1145/2996466 · Cited by 48
Warshaw, J., Matthews, T., Whittaker, S., Kau, C., Bengualid, M., & Smith, B. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702274 · Cited by 47
Tene, O., & Polonetsky, J. (2011) · SSRN Electronic Journal · doi:10.2139/ssrn.1920505 · Cited by 34
Dice in the Black Box: User Experiences with an Inscrutable Algorithm
Springer, A., Hollis, V., & Whittaker, S. (2018) · arXiv · doi:10.48550/arxiv.1812.03219 · Cited by 29
MyTrackingChoices: Pacifying the Ad-Block War by Enforcing User Privacy PreferencesPreprint
Achara, J. P., Parra‐Arnau, J., & Castelluccia, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.04495 · Cited by 24
The Case of the Strangerationist: Re-interpreting Critical Technical Practice
Khovanskaya, V. D., Bezaitis, M., & Sengers, P. (2016) · Designing Interactive Systems Conference · doi:10.1145/2901790.2901860 · Cited by 19
Data, Trust, and Transparency in Personalized Advertising.
Stevenson, D. M. (2016) · Deep Blue (University of Michigan) · Cited by 11
Research works printed but not linked: 11
- Howard Beales. 2010. The value of behavioral targeting. Network Advertising Initiative 1 (2010).
- Matthew Chalmers and Ian MacColl. 2003. Seamful and seamless design in ubiquitous computing. In Workshop at the crossroads: The interaction of HCI and systems issues in UbiComp, Vol. 8.
- Nicholas Diakopoulos. 2014. Algorithmic-Accountability: the investigation of Black Boxes. Tow Center for Digital Journalism (2014).
- Kashmir Hill. 2017. Facebook Figured Out My Family Secrets, And It Won't Tell Me How. (2017). http://bit.ly/2xjBorJ.
- Terry Parris Jr. Julia Angwin and Surya Mattu. 2016. BREAKING THE BLACK BOX: What Facebook Knows About You. (2016). https://www.propublica.org/article/ breaking-the-black-box-what-facebook-knows-about-you.
- 10.5555/1822585 · doi:10.5555/1822585
- 10.5555/2717112 · doi:10.5555/2717112
- Ashwini Rao, Florian Schaub, and Norman Sadeh. 2015. What do they know about me? Contents and concerns of online behavioral profiles. arXiv preprint arXiv:1506.01675 (2015). · arXiv
- Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and discrimination: converting critical concerns into productive inquiry (2014).
- Nick Seaver. 2013. Knowing algorithms. Media in Transition 8 (2013), 1--12.
- WhiteHouse. 2016. Big Data: A Report on Algorithmic Systems, Opportunity, and Civil Rights. Washington, DC: Executive Office of the President, White House (2016).
Cited by
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728
“tered approach, building on previous psychology and social science research on human trust in models [4, 17, 21] and similar efforts in the machine learning and human-computer interaction communities [2, 6, 8, 13, 15, 23, 24]. We present a sequence of large-scale, randomized human-subject experiments in which we vary…”
BackgroundCai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594
“Domains of study include recommender systems [4], medicine [12, 75], social media [22], creativity [14], and advertisements [23].”
BackgroundTowards transparency by design for artificial intelligence
Felzmann, H., Fosch-Villaronga, E., Lutz, C., & Tamò-Larrieux, A. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00276-4 · Cited by 588
“Too much transparency can be undesirable or even harmful (Ananny and Crawford 2018) and empirical research has shown that too much transparency can have unintended side-effects such as algorithmic disillusionment (Eslami et al. 2018).”
Background
Metadata
- Authors
- Eslami, M., Krishna Kumaran, S. R., Sandvig, C., & Karahalios, K.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- University of Illinois at Urbana Champaign, Urbana, IL, USA
- University of Michigan (US)
- Venue
- Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-21
- Pages
- 1-13
- OpenAlex
- W2796410451
- Cited by
- 171
- References
- 44
- Citations by year
- 2026: 11 · 2025: 19 · 2024: 20 · 2023: 14 · 2022: 33 · 2021: 24 · 2020: 23 · 2019: 22 · 2018: 5
- FWCI
- 56.37
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- NSF grant (CHS-1564041)
- Google PhD Fellowship
- Topics
- Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- online behavioral advertising, algorithmic transparency, algorithmic explanations, user trust, algorithmic opacity, user privacy, explainable AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.