Trust in Public Policy Algorithms
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Algorithms are playing an increasingly important role in many areas of public policy. Yet, we know surprisingly little about the degree of trust individuals are willing to place in these algorithms. This article reports on a series of experiments on trust in algorithms for forecasting political events and criminal recidivism. Contrary to previous literature on algorithm aversion, we find that people show high levels of trust in algorithms relative to other sources of advice, even with minimal information about the algorithm. We also explore evaluation of combined human and algorithm advice. We find that, when experts make decisions in light of algorithm advice, the relative weight given to their judgments increases, but the algorithm’s guidance is not disregarded. Finally, using a conjoint experiment, we evaluate the factors that influence people’s preferences for algorithms, finding that risk aversion, data size, human in the loop, and developer reputation play an important role in engendering trust.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Framing of Decisions and the Psychology of Choice
Tversky, A., & Kahneman, D. (1981) · Science · doi:10.1126/science.7455683 · Cited by 18,497
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554
Kim, D. J., Ferrin, D. L., & Rao, H. R. (2008) · Decision Support Systems · doi:10.1016/j.dss.2007.07.001 · Cited by 3,613 · cited 1×
“These include statistical models for forecasting state failure (Goldstone et al. 2010; King and Zeng 2001), political protests (Barberá et al. 2015; Rama-krishnan et al. 2014), international and domestic con fl ict (Gleditsch and Ward 2013; Hegre et al. 2013; O ’ Brien 2010), genocide reports…”
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
The Parable of Google Flu: Traps in Big Data Analysis
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014) · Science · doi:10.1126/science.1248506 · Cited by 2,523
Self-citationAlgorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Hainmueller, J., Hopkins, D. J., & Yamamoto, T. (2014) · Political Analysis · doi:10.1093/pan/mpt024 · Cited by 1,990 · cited 1×
“…and Zeng 2001), political protests (Barberá et al. 2015; Rama-krishnan et al. 2014), international and domestic con fl ict (Gleditsch and Ward 2013; Hegre et al. 2013; O ’ Brien 2010), genocide reports (Goldsmith et al. 2013), and election outcomes (Kennedy, Wojcik, and Lazer 2017; Silver 2012),…”
BackgroundThe robust beauty of improper linear models in decision making.
Dawes, R. M. (1979) · American Psychologist · doi:10.1037/0003-066x.34.7.571 · Cited by 1,966
Validating vignette and conjoint survey experiments against real-world behavior
Hainmueller, J., Hangartner, D., & Yamamoto, T. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1416587112 · Cited by 1,344
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162
The accuracy, fairness, and limits of predicting recidivism
Dressel, J., & Farid, H. (2018) · Science Advances · doi:10.1126/sciadv.aao5580 · Cited by 1,071
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
Coppock, A., & McClellan, O. A. (2019) · Research & Politics · doi:10.1177/2053168018822174 · Cited by 987
The Critical Periphery in the Growth of Social Protests
Barberá, P., Wang, N., Bonneau, R., Jost, J. T., Nagler, J., Tucker, J., & González-Bailón, S. (2015) · PLOS ONE · doi:10.1371/journal.pone.0143611 · Cited by 348 · cited 1×
“…gures in the development of personal computing in the 1960s and 1970s, argued that humans and computers needed to undergo a process of “ bootstrapping, ” whereby humans and machines would undergo “ coevolution ” to produce a system that was more effective than either could be alone (Bardini 2000).”
BackgroundPsychological Strategies for Winning a Geopolitical Forecasting Tournament
Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., Scott, S. E., Moore, D., Atanasov, P., Swift, S. A., et al. (2014) · Psychological Science · doi:10.1177/0956797614524255 · Cited by 312 · cited 1×
“There have also been efforts to improve human-based forecasting by applying algorithms to aggregate group forecasts (Mellers et al. 2014; Tetlock and Gardner 2016).”
BackgroundRisk Assessment in Criminal Sentencing
Monahan, J., & Skeem, J. L. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-092945 · Cited by 256
ViEWS: A political violence early-warning system
Hegre, H., Allansson, M., Basedau, M., Colaresi, M., Croicu, M., Fjelde, H., Hoyles, F., Hultman, L., Högbladh, S., Jansen, R., et al. (2019) · Journal of Peace Research · doi:10.1177/0022343319823860 · Cited by 173 · cited 1×
“A range of algorithms from the PITF models (Goldstone et al. 2010) to the EMBERS (Ramakrishnan et al. 2014) and ViEWS (Hegre et al. 2019) platforms are designed to be used by intelligence of fi cers, government decision-makers, and, in some cases, the public.”
Method
Cited by
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527
Laux, J., Wachter, S., & Mittelstadt, B. (2024) · Regulation & Governance · doi:10.1111/rego.12512 · Cited by 365
Advice from artificial intelligence: a review and practical implications
Baines, J. I., Dalal, R. S., Ponce, L. P., & Tsai, H.-C. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1390182 · Cited by 28
Liu, H. K., Tang, M., & Collard, A. S. J. (2025) · Government Information Quarterly · doi:10.1016/j.giq.2024.102006 · Cited by 17
The Role of Personality in Trust in Public Policy Automation
Waggoner, P., & Kennedy, R. (2022) · Journal of Behavioral Data Science · doi:10.35566/v2n1/p4/ · Cited by 7
Self-citation
Metadata
- Authors
- Kennedy, R. P., Waggoner, P. D., & Ward, M. M.
- Venue
- The Journal of Politics
- Publisher
- University of Chicago Press
- Published
- 2022-04-01
- Volume
- 84
- Issue
- 2
- Pages
- 1132-1148
- ISSN
- 0022-3816 (ISSN-L)
- 1468-2508
- OpenAlex
- W2981041627
- Cited by
- 46
- References
- 21
- FWCI
- 2.93
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.