Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them
Open accessAbstract
Although evidence-based algorithms consistently outperform human forecasters, people often fail to use them after learning that they are imperfect, a phenomenon known as algorithm aversion. In this paper, we present three studies investigating how to reduce algorithm aversion. In incentivized forecasting tasks, participants chose between using their own forecasts or those of an algorithm that was built by experts. Participants were considerably more likely to choose to use an imperfect algorithm when they could modify its forecasts, and they performed better as a result. Notably, the preference for modifiable algorithms held even when participants were severely restricted in the modifications they could make (Studies 1–3). In fact, our results suggest that participants’ preference for modifiable algorithms was indicative of a desire for some control over the forecasting outcome, and not for a desire for greater control over the forecasting outcome, as participants’ preference for modifiable algorithms was relatively insensitive to the magnitude of the modifications they were able to make (Study 2). Additionally, we found that giving participants the freedom to modify an imperfect algorithm made them feel more satisfied with the forecasting process, more likely to believe that the algorithm was superior, and more likely to choose to use an algorithm to make subsequent forecasts (Study 3). This research suggests that one can reduce algorithm aversion by giving people some contr…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Allowing users to adjust the algorithm can reduce resistance.
“At the same time, some design choices can reduce resistance -for example, allowing users to adjust the algorithm (Dietvorst et al., 2018)”
From Algorithm aversion in academic libraries: User resistance to AI-enabled discovery and search services (2026) · cites this paper as Dietvorst et al., 2018
Being empowered to change or intervene in an algorithmic decision can overcome the aversion reaction sometimes developed by algorithm users.
“A lab study by Dietvorst, Simmons, and Massey (2016) showed that being empowered to change or intervene in an algorithmic decision can overcome the aversion reaction sometimes developed by algorithm' users.”
From Algorithms as work designers: How algorithmic management influences the design of jobs (2022) · cites this paper as Dietvorst, Simmons, and Massey, 2016
Even a very small amount of power of intervention or modification can significantly modify one's reaction and performance when working with the algorithm.
“Their results further indicate that even a very small amount of power of intervention or modification can significantly modify one's reaction and performance when working with the algorithm.”
From Algorithms as work designers: How algorithmic management influences the design of jobs (2022) · cites this paper as Dietvorst, Simmons, and Massey, 2016
Cites
Preacher, K. J., & Hayes, A. F. (2008) · Behavior Research Methods · doi:10.3758/brm.40.3.879 · Cited by 34,117 · cited 1× · see citing works
“We then used Preacher and Hayes’s (2008) bootstrapping procedure”
MethodAlgorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 6× · see citing works
“Dietvorst, Simmons, and Massey (2015) gave participants the choice of either exclusively using an algorithm’s forecasts or exclusively using their own forecasts during an incentivized forecasting task, and manipulated whether participants had experience with the algorithm prior to making this…”
BackgroundResultInfluentialSelf-citation“Clinical versus Actuarial Judgment.”
Dawes, R. M., Faust, D., & Meehl, P. E. (1989) · Science · doi:10.1126/science.2648573 · Cited by 2,298 · see citing works
Clinical versus statistical prediction: A theoretical analysis and a review of the evidence.
Meehl, P. E. (1954) · University of Minnesota Press eBooks · doi:10.1037/11281-000 · Cited by 2,187 · see citing works
The robust beauty of improper linear models in decision making.
Dawes, R. M. (1979) · American Psychologist · doi:10.1037/0003-066x.34.7.571 · Cited by 1,970 · see citing works
Clinical versus mechanical prediction: a meta-analysis
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000) · Psychological Assessment · doi:10.1037/1040-3590.12.1.19 · Cited by 1,640 · cited 2× · see citing works
“…of recidivism (Thompson, 1952; Wormith & Goldstone, 1984), medical diagnoses (Adams et al., 1986; Beck et al., 2011; Dawes, Faust, & Meehl, 1989; Grove et al., 2000), demand for products (Schweitzer & Cachon, 2000), and so on (for reviews see Dawes et al., 1989; Grove et al., 2000; Meehl, 1954).”
BackgroundDecision Bias in the Newsvendor Problem with a Known Demand Distribution: Experimental Evidence
Schweitzer, M. E., & Cachon, G. P. (2000) · Management Science · doi:10.1287/mnsc.46.3.404.12070 · Cited by 1,128 · cited 2× · see citing works
“…of recidivism (Thompson, 1952; Wormith & Goldstone, 1984), medical diagnoses (Adams et al., 1986; Beck et al., 2011; Dawes, Faust, & Meehl, 1989; Grove et al., 2000), demand for products (Schweitzer & Cachon, 2000), and so on (for reviews see Dawes et al., 1989; Grove et al., 2000; Meehl, 1954).”
BackgroundBonaccio, S., & Dalal, R. S. (2006) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2006.07.001 · Cited by 1,011 · cited 1× · see citing works
“For example, people often give less weight to other people’s advice than they should when left to their own devices (see Bonaccio & Dalal, 2006); however, it is possible that people will commit to using another person’s advice in a constrained manner.”
BackgroundSystematic Analysis of Breast Cancer Morphology Uncovers Stromal Features Associated with Survival
Beck, A. H., Sangoi, A. R., Leung, S. C. Y., Marinelli, R. J., Nielsen, T. O., van de Vijver, M. J., West, R. B., van de Rijn, M., & Koller, D. (2011) · Science Translational Medicine · doi:10.1126/scitranslmed.3002564 · Cited by 765 · cited 1× · see citing works
“…(Dawes, 1971; Dawes, 1979), prisoners’ likelihood of recidivism (Thompson, 1952; Wormith & Goldstone, 1984), medical diagnoses (Adams et al., 1986; Beck et al., 2011; Dawes, Faust, & Meehl, 1989; Grove et al., 2000), demand for products (Schweitzer & Cachon, 2000), and so on (for reviews see…”
Forecasting and Planning: An Evaluation
Hogarth, R. M., & Makridakis, S. (1981) · Management Science · doi:10.1287/mnsc.27.2.115 · Cited by 610 · cited 2× · see citing works
“…people’s attempts to adjust algorithmic forecasts often make them worse (e.g. Carbone, Andersen, Corriveau, & Corson, 1983; Goodwin & Fildes, 1999; Hogarth & Makridakis, 1981; Lim & O'Connor, 1995; Willemain, 1991), the benefits associated with getting people to use the algorithm may outweigh the…”
BackgroundA case study of graduate admissions: Application of three principles of human decision making.
Dawes, R. M. (1971) · American Psychologist · doi:10.1037/h0030868 · Cited by 578 · cited 1× · see citing works
“…of research, has been observed in many different domains, including forecasts of employee performance (see Highhouse, 2008), academic performance (Dawes, 1971; Dawes, 1979), prisoners’ likelihood of recidivism (Thompson, 1952; Wormith & Goldstone, 1984), medical diagnoses (Adams et al., 1986;…”
BackgroundJudgmental forecasting: A review of progress over the last 25years
Lawrence, M. J., Goodwin, P., O’Connor, M. J., & Önkal, D. (2006) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2006.03.007 · Cited by 534 · see citing works
Stubborn Reliance on Intuition and Subjectivity in Employee Selection
Highhouse, S. (2008) · Industrial and Organizational Psychology · doi:10.1111/j.1754-9434.2008.00058.x · Cited by 498 · cited 2× · see citing works
“…regularity, supported by decades of research, has been observed in many different domains, including forecasts of employee performance (see Highhouse, 2008), academic performance (Dawes, 1971; Dawes, 1979), prisoners’ likelihood of recidivism (Thompson, 1952; Wormith & Goldstone, 1984),…”
BackgroundFildes, R. A., Goodwin, P., Lawrence, M. J., & Nikolopoulos, K. I. (2009) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2008.11.010 · Cited by 447 · see citing works
Factors influencing the use of a decision rule in a probabilistic task
Arkes, H. R., Dawes, R. M., & Christensen, C. (1986) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(86)90046-4 · Cited by 333 · see citing works
Computer aided diagnosis of acute abdominal pain: a multicentre study.
Adams, I. D., Chan, M. Q., Clifford, P. C., Cooke, W. M., Dallos, V., de Dombal, F. T., Edwards, M., Hancock, D. M., Hewett, D. J., & McIntyre, N. (1986) · BMJ · doi:10.1136/bmj.293.6550.800 · Cited by 311 · cited 5× · see citing works
“1 This empirical regularity, supported by decades of research, has been observed in many different domains, including forecasts of employee performance (see Highhouse 2008), academic performance (Dawes 1971, 1979), prisoners’ likelihood of recidivism (Thompson 1952, Wormith and Goldstone 1984), medical diagnoses (Adam…”
BackgroundInfluentialFildes, R. A., & Goodwin, P. (2007) · INFORMS Journal on Applied Analytics · doi:10.1287/inte.1070.0309 · Cited by 225 · see citing works
Accepting Error to Make Less Error
Einhorn, H. J. (1986) · Journal of Personality Assessment · doi:10.1207/s15327752jpa5003_8 · Cited by 179 · cited 1× · see citing works
“Einhorn (1986) proposed that forecasters’ intolerance of algorithms arises because although people believe that algorithms will necessarily err, they believe that humans are capable of perfection (also see Highhouse, 2008).”
BackgroundJudgemental adjustment of initial forecasts: Its effectiveness and biases
Lim, J. S., & O’Connor, M. J. (1995) · Journal of Behavioral Decision Making · doi:10.1002/bdm.3960080302 · Cited by 172 · see citing works
The efficacy of using judgmental versus quantitative forecasting methods in practice
Sanders, N. R., & Manrodt, K. B. (2003) · Omega · doi:10.1016/j.omega.2003.08.007 · Cited by 148 · cited 1× · see citing works
“Sanders and Manrodt (2003) surveyed 240 firms and found that many did not use algorithms for forecasting, and that firms that did use algorithms made fewer forecasting errors.”
BackgroundThe Clinical and Statistical Prediction of Recidivism
Wormith, J. S., & GOLDSTONE, C. S. (1984) · Criminal Justice and Behavior · doi:10.1177/0093854884011001001 · Cited by 84 · cited 1× · see citing works
“…of employee performance (see Highhouse, 2008), academic performance (Dawes, 1971; Dawes, 1979), prisoners’ likelihood of recidivism (Thompson, 1952; Wormith & Goldstone, 1984), medical diagnoses (Adams et al., 1986; Beck et al., 2011; Dawes, Faust, & Meehl, 1989; Grove et al., 2000), demand for…”
Carbone, R., Andersen, A., Corriveau, Y., et al. (1983) · Management Science · doi:10.1287/mnsc.29.5.559 · Cited by 77 · cited 1× · see citing works
“3 Although people’s attempts to adjust algorithmic forecasts often make them worse (e.g. Carbone, Andersen, Corriveau, & Corson, 1983; Goodwin & Fildes, 1999; Hogarth & Makridakis, 1981; Lim & O'Connor, 1995; Willemain, 1991), the benefits associated with getting people to use the algorithm may…”
BackgroundSurvey on the use of clinical and mechanical prediction methods in clinical psychology.
Vrieze, S. I., & Grove, W. M. (2009) · Professional Psychology Research and Practice · doi:10.1037/a0014693 · Cited by 59 · cited 1× · see citing works
“Vrieze and Grove (2009) surveyed 183 clinical psychologists and found that only 31% of them used algorithms when making clinical predictions.”
BackgroundOrder effects and memory for evidence in individual versus group decision making in auditing
Ahlawat, S. S. (1999) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199903)12:1 · Cited by 55 · see citing works
The effect of graphical adjustment on forecast accuracy
Willemain, T. R. (1991) · International Journal of Forecasting · doi:10.1016/0169-2070(91)90049-2 · Cited by 33 · cited 1× · see citing works
“…often make them worse (e.g. Carbone, Andersen, Corriveau, & Corson, 1983; Goodwin & Fildes, 1999; Hogarth & Makridakis, 1981; Lim & O'Connor, 1995; Willemain, 1991), the benefits associated with getting people to use the algorithm may outweigh the costs associated with degrading the algorithm’s…”
BackgroundA Validation of the Glueck Social Prediction Scale for Proneness to Delinquency
Thompson, R. E. (1952) · The Journal of Criminal Law Criminology and Police Science · doi:10.2307/1139334 · Cited by 17 · cited 1× · see citing works
“…forecasts of employee performance (see Highhouse, 2008), academic performance (Dawes, 1971; Dawes, 1979), prisoners’ likelihood of recidivism (Thompson, 1952; Wormith & Goldstone, 1984), medical diagnoses (Adams et al., 1986; Beck et al., 2011; Dawes, Faust, & Meehl, 1989; Grove et al.,…”
Background
Flagged first, then by citations
Cited by
Sorted by citations
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · see citing works
“For instance, Dzindolet et al. (2003) tested an automated decision aid and found that errors significantly decreased trust and reliance on the aid.”
BackgroundBrave new world: service robots in the frontline
Wirtz, J., Patterson, P. G., Kunz, W. H., Gruber, T., Lu, V. N., Paluch, S., & Martins, A. (2018) · Journal of Service Management · doi:10.1108/josm-04-2018-0119 · Cited by 2,315 · see citing works
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061 · see citing works
Resistance to Medical Artificial Intelligence
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019) · Journal of Consumer Research · doi:10.1093/jcr/ucz013 · Cited by 2,015 · see citing works
Task-Dependent Algorithm Aversion
Castelo, N., Bos, M. W., & Lehmann, D. R. (2019) · Springer eBooks · doi:10.1007/978-3-031-88067-4_21 · Cited by 1,596 · see citing works
Consumers and artificial intelligence: An experiential perspective
Puntoni, S., Reczek, R. W., Giesler, M., & Botti, S. (2021) · Journal of Marketing · doi:10.1177/0022242920953847 · Cited by 1,193 · see citing works
“For instance, offering users the possibility to correct an algorithm’s output, even if only slightly, is enough to increase their likelihood of using the superior, although imperfect, algorithm rather than the preferred, inferior human forecast (Dietvorst, Simmons, and Massey 2016).”
Luo, X., Tong, S., Fang, Z., & Qu, Z. (2019) · Marketing Science · doi:10.1287/mksc.2019.1192 · Cited by 1,027 · see citing works
Celik, I. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107468 · Cited by 1,012 · see citing works
Forecasting: theory and practice
Petropoulos, F., Apiletti, D., Assimakopoulos, V., Babai, M. Z., Barrow, D. K., Taieb, S. B., Bergmeir, C., Bessa, R., Bijak, J., Boylan, J., et al. (2020) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.11.001 · Cited by 630 · see citing works
“Prahl and Van Swol, 2017). Such ‘algorithm aversion’ and error intolerance is reduced when users have opportunity to adjust the forecasting outcome, irrespective of the extent of modification allowed (Dietvorst et al., 2018). Feedback appears to improve trust, with individuals placing higher trust in algorithms if they…”
BackgroundAutomation, Algorithms, and Beyond: Why Work Design Matters More Than Ever in a Digital World
Parker, S. K., & Grote, G. (2022) · Applied Psychology · doi:10.1111/apps.12241 · Cited by 960 · see citing works
Artificial Intelligence in Utilitarian vs. Hedonic Contexts: The “Word-of-Machine” Effect
Longoni, C., & Cian, L. (2022) · Journal of Marketing · doi:10.1177/0022242920957347 · Cited by 831 · see citing works
“…circumstances resistance to algorithmic advice is attenuated: if people have the opportunity to modify algorithms and thus exert control over them (Dietvorst, Simmons, and Massey 2016), if the human likeness of algorithms is increased (Castelo, Bos, and Lehman 2019), if the task entails a numeric…”
BackgroundResultAlgorithmic bias: review, synthesis, and future research directions
Kordzadeh, N., & Ghasemaghaei, M. (2022) · European Journal of Information Systems · doi:10.1080/0960085x.2021.1927212 · Cited by 770 · see citing works
“These courses of actions or behavioural responses include refraining from accepting the algorithmic recommendation (Ebrahimi & Hassanein, 2019; Grgić-Hlača et al., 2019), engaging in algorithm aversion (Dietvorst et al., 2018), and refusing to adopt an algorithmic system (Lee, Jain et al., 2019).”
A systematic review of algorithm aversion in augmented decision making
Burton, J. W., Stein, M., & Jensen, T. B. (2020) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2155 · Cited by 814 · see citing works
“Could the reduction in algorithm aversion caused by allowing human users to modify the algorithmic aid's output (e.g., Dietvorst et al., 2016) be fur-thered by allowing them to also modify the algorithmic aid's input?”
BackgroundInfluentialManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · see citing works
Industry 4.0: Opportunities and Challenges for Operations Management
Olsen, T. L., & Tomlin, B. T. (2019) · Manufacturing & Service Operations Management · doi:10.1287/msom.2019.0796 · Cited by 651 · see citing works
What influences algorithmic decision-making? A systematic literature review on algorithm aversion
Mahmud, H., Islam, A. K. M. N., Ahmed, S. I., & Smolander, K. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.121390 · Cited by 601 · see citing works
Newman, D. T., Fast, N. J., & Harmon, D. J. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.03.008 · Cited by 550 · see citing works
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548 · see citing works
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · see citing works
The Promises and Pitfalls of Robo-Advising
D’Acunto, F., Prabhala, N. R., & Rossi, A. G. P. (2019) · Review of Financial Studies · doi:10.1093/rfs/hhz014 · Cited by 585 · see citing works
Baird, A., & Maruping, L. M. (2021) · MIS Quarterly · doi:10.25300/misq/2021/15882 · Cited by 512 · see citing works
“2000), algorithm aversion (e.g., Burton et al. 2019; Dietvorst et al. 2016), agent-based and multi-agent systems (e.”
BackgroundMethodKim, S. Y., Schmitt, B. H., & Thalmann, N. M. (2019) · Marketing Letters · doi:10.1007/s11002-019-09485-9 · Cited by 498 · see citing works
Algorithmic management in a work context
Jarrahi, M. H., Newlands, G., Lee, M. K., Wolf, C. T., Kinder, E., & Sutherland, W. (2021) · Big Data & Society · doi:10.1177/20539517211020332 · Cited by 532 · see citing works
“Algorithm aversion refers to people’s hesitance to use algorithmic results, usually after observing imperfect performance by algorithms; this can even occur in situations where the algorithm may outperform humans in certain accuracy measures (Dietvorst et al., 2016; Prahl and Van Swol, 2017).”
BackgroundFügener, A., Grahl, J., Gupta, A., & Ketter, W. (2021) · Information systems research · doi:10.1287/isre.2021.1079 · Cited by 474 · see citing works
Algorithms as work designers: How algorithmic management influences the design of jobs
Parent-Rocheleau, X., & Parker, S. K. (2022) · Human Resource Management Review · doi:10.1016/j.hrmr.2021.100838 · Cited by 512 · see citing works
Making sense of recommendations
Yeomans, M., Shah, A., Mullainathan, S., & Kleinberg, J. (2019) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2118 · Cited by 459 · see citing works
Transparency and trust in artificial intelligence systems
Schmidt, P., Biessmann, F., & Teubner, T. (2020) · Journal of Decision Systems · doi:10.1080/12460125.2020.1819094 · Cited by 440 · see citing works
“In follow-up work, B. Dietvorst et al. (2018) showed that people were more than twice as likely to rely on algorithmic forecasts if they were allowed to change a fraction of the AIs predictions compared to when they were bound what the AI predicted.”
BackgroundJussupow, E., Spohrer, K., Heinzl, A., & Gawlitza, J. (2021) · Information Systems Research · doi:10.1287/isre.2020.0980 · Cited by 422 · see citing works
“A second research stream in IS has suggested that distinct heuristic processes of human decision makers prevent adequate advice utilization (Xiao and Benbasat 2011, 2015, Dietvorst et al. 2015, 2018).”
BackgroundFranke, C., Groeppel-Klein, A., & Müller, K. (2023) · Journal of Advertising · doi:10.1080/00913367.2022.2154721 · Cited by 408 · see citing works
“This could be owing to a phenomenon called algorithm aversion: individuals tend to have a general antipathy toward algorithms, particularly in the case of algorithms making mistakes (Dietvorst, Simmons, and Massey 2018).”
BackgroundLeveraging Artificial Intelligence in Marketing for Social Good—An Ethical Perspective
Hermann, E. (2021) · Journal of Business Ethics · doi:10.1007/s10551-021-04843-y · Cited by 458 · see citing works
Estimating the Impact of “Humanizing” Customer Service Chatbots
Schanke, S., Burtch, G., & Ray, G. (2021) · Information Systems Research · doi:10.1287/isre.2021.1015 · Cited by 407 · see citing works
AI-Based Digital Assistants: Opportunities, Threats, and Research Perspectives
Maedche, A., Legner, C., Benlian, A., Berger, B., Gimpel, H., Hess, T., Hinz, O., Morana, S., & Söllner, M. (2019) · Business & Information Systems Engineering · doi:10.1007/s12599-019-00600-8 · Cited by 432 · see citing works
Singh, J., Flaherty, K. E., Sohi, R. S., Deeter‐Schmelz, D. R., Habel, J., Meunier‐FitzHugh, K. L., Malshe, A., Mullins, R. R., & Onyemah, V. (2019) · Journal of Personal Selling and Sales Management · doi:10.1080/08853134.2018.1557525 · Cited by 366 · see citing works
“The implementation of AI often results in changes in jobs due to the redistribution of tasks between humans and robots (Edwards 1984).”
BackgroundSong, M., Xing, X., Duan, Y., Cohen, J. F., & Mou, J. (2022) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2021.102900 · Cited by 345 · see citing works
Artificial Intelligence Coaches for Sales Agents: Caveats and Solutions
Luo, X., Qin, M. S., Fang, Z., & Qu, Z. (2020) · Journal of Marketing · doi:10.1177/0022242920956676 · Cited by 338 · see citing works
Yam, K. C., Tang, P. M., Jackson, J. C., Su, R., & Gray, K. J. (2022) · Journal of Applied Psychology · doi:10.1037/apl0001045 · Cited by 352 · see citing works
Trusting Automation: Designing for Responsivity and Resilience
Chiou, E. K., & Lee, J. D. (2023) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211009995 · Cited by 301 · see citing works
“guide its estimate of trust, to not just calibrate trust, but also to reframe trust, such as prompting people to exert a little control over the automation to counteract algorithm aversion (Dietvorst et al., 2018).”
BackgroundBenbya, H., Pachidi, S., & Järvenpää, S. L. (2021) · Journal of the Association for Information Systems · doi:10.17705/1jais.00662 · Cited by 340 · see citing works
“Compared to tasks requiring social or emotional intelligence, individuals appear to be more trusting of AI tools for technical tasks that require complex processing such as data analysis (Salem et al., 2015; Dietvorst et al., 2016; Dzindolet et al., 2003).”
Human-machine collaboration in managerial decision making
Haesevoets, T., De Cremer, D., Dierckx, K., & Van Hiel, A. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106730 · Cited by 280 · see citing works
Understanding algorithm aversion: When is advice from automation discounted?
Prahl, A., & Van Swol, L. (2017) · Journal of Forecasting · doi:10.1002/for.2464 · Cited by 308 · see citing works
Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments
Green, B., & Chen, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287563 · Cited by 275 · see citing works
“One study found that people could not distinguish between reliable and unreliable predictions [30], and another found that people often deviate incorrectly from algorithmic fore-casts [18].”
BackgroundIntelligent digital twin (iDT) for supply chain stress-testing, resilience, and viability
Ivanov, D. (2023) · International Journal of Production Economics · doi:10.1016/j.ijpe.2023.108938 · Cited by 241 · see citing works
Expl(AI)ned: The Impact of Explainable Artificial Intelligence on Users’ Information Processing
Bauer, K., von Zahn, M., & Hinz, O. F. (2023) · Information Systems Research · doi:10.1287/isre.2023.1199 · Cited by 268 · see citing works
“…subjectivity of the task (Castelo et al. 2019, Logg et al. 2019), seeing the system err (Dietvorst et al. 2015), being able to modify predictions (Dietvorst et al. 2018), the divergence bet-ween actual and expected predictive performance (Jus-supow et al. 2020), and, most importantly for our…”
BackgroundAutonomous Shopping Systems: Identifying and Overcoming Barriers to Consumer Adoption
de Bellis, E., & Johar, G. V. (2020) · Journal of Retailing · doi:10.1016/j.jretai.2019.12.004 · Cited by 270 · see citing works
Kang, H., & Lou, C. (2022) · J. Comput. Mediat. Commun. · doi:10.1093/jcmc/zmac014 · Cited by 250 · see citing works
Beware explanations from AI in health care
Babic, B., Gerke, S., Evgeniou, T., & Cohen, I. G. (2021) · Science · doi:10.1126/science.abg1834 · Cited by 252 · see citing works
Hegner, S. M., Beldad, A. D., & Brunswick, G. J. (2019) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2019.1572353 · Cited by 249 · see citing works
How transparency modulates trust in artificial intelligence
Zerilli, J., Bhatt, U., & Weller, A. (2022) · Patterns · doi:10.1016/j.patter.2022.100455 · Cited by 257 · see citing works
“5,14,15 Users then typically retreat to human judgment, even when doing so leads demonstrably to even more errors.”
BackgroundInfluentialExperimental evidence of effective human–AI collaboration in medical decision-making
Reverberi, C., Rigon, T., Solari, A., Hassan, C., Cherubini, P., GI Genius CADx Study Group, Antonelli, G., Awadie, H., Bernhofer, S., Carballal, S., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-18751-2 · Cited by 235 · see citing works
Artificial intelligence and consumer behavior: From predictive to generative AI
Hermann, E., & Puntoni, S. (2024) · Journal of Business Research · doi:10.1016/j.jbusres.2024.114720 · Cited by 284 · see citing works
Alabed, A., Javornik, A., & Gregory-Smith, D. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.121786 · Cited by 233 · see citing works
Value-Sensitive Algorithm Design: Method, Case Study, and Lessons
Zhu, H., Yu, B., Halfaker, A., & Terveen, L. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274463 · Cited by 220 · see citing works
“We measured the effect of our system on new editors, specifically in terms of the number of their edits within the relevant WikiProjects and in the rest of Wikipedia 16 .”
MethodA Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
De-Arteaga, M., Fogliato, R., & Chouldechova, A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376638 · Cited by 221 · see citing works
“Meanwhile, studies focused on the causes of algorithm aversion have found that repeatedly seeing the algorithm make the same mistake leads to decreased reliance of the agent on the system [13], while giving some control over the algorithm can counter this phenomenon [14].”
BackgroundPsychological factors underlying attitudes toward AI tools
Freitas, J. D., Agarwal, S., Schmitt, B. H., & Haslam, N. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01734-2 · Cited by 226 · see citing works
“Granting too much control over AI systems can make decision-making less accurate 91,92 , given that evidence-based AI systems consistently outperform human decision-makers.”
BackgroundDesigning Transparency for Effective Human-AI Collaboration
Vössing, M., Kühl, N., Lind, M., & Satzger, G. (2022) · Information Systems Frontiers · doi:10.1007/s10796-022-10284-3 · Cited by 266 · see citing works
“As noted by Dietvorst et al. (2018), the “reluctance to use algorithms for forecasting [often] stems from an intolerance of inevitable error” (p. 1156).”
BackgroundResultThree Challenges for AI-Assisted Decision-Making
Steyvers, M., & Kumar, A. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916231181102 · Cited by 217 · see citing works
“For example, Dietvorst et al. (2018) showed that allowing individuals to intervene in an algorithm’s decision caused the individuals in question to appreciate the algorithm more and made them more likely to use the algorithm in the future even though allowing the algo-rithm to make the decision…”
Automated rationale generation: a technique for explainable AI and its effects on human perceptions
Ehsan, U., Tambwekar, P., Chan, L., Harrison, B., & Riedl, M. O. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302316 · Cited by 210 · see citing works
“For instance, contestability, the ability to either reject a reasoning or ask for another one, which has shown to improve user satisfactions [16, 22] can be incorporated in the future.”
BackgroundWanner, J., Herm, L.-V., Heinrich, K., & Janiesch, C. (2022) · Electronic Markets · doi:10.1007/s12525-022-00593-5 · Cited by 240 · see citing works
“In line with Dietvorst et al. (2016), it is important to manage expectations and show the user control opportunities of the system.”
BackgroundWang, R., Harper, F. M., & Zhu, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376813 · Cited by 211 · see citing works
“The aversion to algorithmic decision-making could be mitigated when users contribute to the modification of the algorithms [24].”
BackgroundThe transparency dilemma: How AI disclosure erodes trust
Schilke, O. S., & Reimann, M. (2025) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2025.104405 · Cited by 265 · see citing works
“One might suspect that the AI-disclosure effect is merely picking up on algorithm aversion, which manifests in more negative attitudes towards an algorithmic versus human agent (e. g., Dietvorst et al., 2018; Newman et al., 2020).”
Dennis, A. R., Lakhiwal, A., & Sachdeva, A. (2023) · Journal of Management Information Systems · doi:10.1080/07421222.2023.2196773 · Cited by 220 · see citing works
“Other research shows that people exhibit algorithm aversion by choosing their judgment over AI’s when they observe imperfections in AI [17, 54], thus indicating bias against AI despite its ability to outperform humans generally [17, 38, 53, 73].”
BackgroundInfluentialKu, K. Y. L., Kong, Q., Song, Y., Deng, L., Kang, Y., & Hu, A. (2019) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2019.05.004 · Cited by 207 · see citing works
Keding, C. (2021) · Management Review Quarterly · doi:10.1007/s11301-020-00181-x · Cited by 231 · see citing works
“…(2017),Wilson and Daugherty (2018), Wright and Schultz (2018) Ghasemaghaei et al. (2018), Kolbjørnsrud et al. (2017), Prahl and van Swol (2017), Dietvorst et al. (2018), Logg et al. (2019), Schneider and Leyer (2019) Organizational determinants Berman and Dalzell-Payne (2018), Bonczek et al.…”
BackgroundLai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204 · see citing works
“…mturk [1, 9, 11, 15, 21, 23, 24, 26, 29, 30, 32, 35, 38, 41, 43–45, 47, 60–62, 71, 73, 74, 79, 82, 83, 86, 91, 94, 110, 112], university [24, 30, 48, 57, 61, 63, 68, 72, 75, 81, 82, 99, 106, 111], prolific [2, 36], company [8, 38, 56, 73, 75, 108], qualitrics [84], local community [12, 58],…”
BackgroundMethodInfluentialOperational Research and Artificial Intelligence Methods in Banking
Doumpos, M., Zopounidis, C., Gounopoulos, D., Platanakis, E., & Zhang, W. (2022) · European Journal of Operational Research · doi:10.1016/j.ejor.2022.04.027 · Cited by 235 · see citing works
Watch Me Improve—Algorithm Aversion and Demonstrating the Ability to Learn
Berger, B., Adam, M., Rühr, A., & Benlian, A. (2020) · Business & Information Systems Engineering · doi:10.1007/s12599-020-00678-5 · Cited by 184 · see citing works
Thumbs Up or Down: Consumer Reactions to Decisions by Algorithms Versus Humans
Williams, G. Y., Lim, S., Puntoni, S., & van Osselaer, S. M. J. (2021) · Journal of Marketing Research · doi:10.1177/00222437211070016 · Cited by 196 · see citing works
Rapid Trust Calibration through Interpretable and Uncertainty-Aware AI
Tomsett, R., Preece, A., Braines, D., Cerutti, F., Chakraborty, S., Srivastava, M., Pearson, G., & Kaplan, L. (2020) · Patterns · doi:10.1016/j.patter.2020.100049 · Cited by 186 · see citing works
Cui, R., Li, M., & Zhang, S. (2020) · Manufacturing & Service Operations Management · doi:10.1287/msom.2021.0989 · Cited by 172 · see citing works
Keding, C., & Meissner, P. (2021) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.120970 · Cited by 168 · see citing works
Human-AI agency in the age of generative AI
Krakowski, S. M. (2025) · Information and Organization · doi:10.1016/j.infoandorg.2025.100560 · Cited by 196 · see citing works
Jago, A. S. (2017) · Academy of Management Discoveries · doi:10.5465/amd.2017.0002 · Cited by 172 · see citing works
From explainable to interactive AI: A literature review on current trends in human-AI interaction
Raees, M., Meijerink, I., Lykourentzou, I., Khan, V.-J., & Papangelis, K. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103301 · Cited by 180 · see citing works
Sun, J., Zhang, D., Hu, H., & Mieghem, J. A. V. (2019) · Management Science · doi:10.1287/mnsc.2021.3990 · Cited by 162 · see citing works
Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology
Agarwal, N., Moehring, A., Rajpurkar, P., & Salz, T. (2023) · doi:10.2139/ssrn.4505053 · Cited by 161 · see citing works
Li, Y., Wu, B., Huang, Y., & Luan, S. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1382693 · Cited by 197 · see citing works
Artificial Intelligence in Human Resources Management: Challenges and a Path ForwardPreprint
Cappelli, P. H., Tambe, P. B., & Yakubovich, V. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3263878 · Cited by 160 · see citing works
Combining Human Expertise with Artificial Intelligence: Experimental Evidence from Radiology
Agarwal, N., Moehring, A., Rajpurkar, P., & Salz, T. (2023) · doi:10.3386/w31422 · Cited by 161 · see citing works
Rise of the machines: Delegating decisions to autonomous AI
Candrian, C., & Scherer, A. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107308 · Cited by 163 · see citing works
Human and Machine: The Impact of Machine Input on Decision Making Under Cognitive Limitations
Boyacı, T., Canyakmaz, C., & de Véricourt, F. (2023) · Management Science · doi:10.1287/mnsc.2023.4744 · Cited by 154 · see citing works
Managing with Artificial Intelligence: An Integrative Framework
Hillebrand, L., Raisch, S., & Schad, J. (2025) · Academy of Management Annals · doi:10.5465/annals.2022.0072 · Cited by 193 · see citing works
Artificial Intelligence, Trust, and Perceptions of Agency
Puranam, P., & Vanneste, B. (2021) · Academy of Management Review · doi:10.5465/amr.2022.0041 · Cited by 188 · see citing works
When Will Workers Follow an Algorithm? A Field Experiment with a Retail Business
Kawaguchi, K. (2018) · Management Science · doi:10.1287/mnsc.2020.3599 · Cited by 155 · see citing works
Algorithmic Decision-Making and the Control Problem
Zerilli, J., Knott, A., Maclaurin, J., & Gavaghan, C. (2019) · Minds and Machines · doi:10.1007/s11023-019-09513-7 · Cited by 145 · see citing works
Zhu, Y., Zhang, J., Wu, J., & Liu, Y. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2022.06.044 · Cited by 148 · see citing works
Prejudiced against the Machine? Implicit Associations and the Transience of Algorithm Aversion
Turel, O., & Kalhan, S. (2023) · MIS Quarterly · doi:10.25300/misq/2022/17961 · Cited by 145 · see citing works
Sohn, K., Sung, E., Koo, G., & Kwon, O. (2020) · International Journal of Retail & Distribution Management · doi:10.1108/ijrdm-03-2020-0091 · Cited by 141 · see citing works
Artificial intelligence, bureaucratic form, and discretion in public service
Bullock, J., Young, M. M., & Wang, Y.-F. (2020) · Information Polity · doi:10.3233/ip-200223 · Cited by 139 · see citing works
Young, M. M., Bullock, J. B., & Lecy, J. D. (2019) · Perspectives on Public Management and Governance · doi:10.1093/ppmgov/gvz014 · Cited by 255 · see citing works
Dual humanness and trust in conversational AI: A person-centered approach
Hu, P., Lu, Y., & Gong, Y. Y. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106727 · Cited by 143 · see citing works
You, S., Yang, C., & Li, X. (2022) · Journal of Management Information Systems · doi:10.1080/07421222.2022.2063553 · Cited by 142 · see citing works
“Castelo et al. [12] and Dietvorst et al. [17] show that, when presented with prediction performance, individuals display higher trust in an algorithm than a human when they can modify their final prediction outcomes based on the advisor’s prediction.”
BackgroundMethodInfluentialHow to overcome algorithm aversion: Learning from mistakes
Reich, T., Kaju, A., & Maglio, S. J. (2022) · Journal of Consumer Psychology · doi:10.1002/jcpy.1313 · Cited by 139 · see citing works
Exploit Prediction Scoring System (EPSS)
Jacobs, J., Romanosky, S., Edwards, B., et al. (2021) · Digital Threats Research and Practice · doi:10.1145/3436242 · Cited by 135 · see citing works
“However, this effect can be overcome if individuals perceive control over the algorithmic inputs [15].”
BackgroundAI and culture: Culturally dependent responses to AI systems
Barnes, A. J., Zhang, Y., & Valenzuela, A. (2024) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2024.101838 · Cited by 135 · see citing works
Making Better Use of the Crowd: How Crowdsourcing Can Advance Machine Learning Research
Vaughan, J. W. (2018) · Journal of Machine Learning Research · Cited by 130 · see citing works
Westphal, M., Vössing, M., Satzger, G., Yom‐Tov, G. B., & Rafaeli, A. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.107714 · Cited by 141 · see citing works
Wang, C., Li, Y., Fu, W., & Jin, J. (2023) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2023.103325 · Cited by 126 · see citing works
Inkpen, K., Chappidi, S., Mallari, K., Nushi, B., Ramesh, D., Michelucci, P., Mandava, V., Vepřek, L. H., & Quinn, G. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3534561 · Cited by 132 · see citing works
“This strategy suggests that individuals are more likely to use and rely on algorithms when they have the opportunity to possibly correct and minimize errors in their decision-making judgement, even when the algorithmic-assistance is imperfect [15].”
BackgroundYam, K. C., Tan, T., Jackson, J. C., Shariff, A., & Gray, K. J. (2023) · Management and Organization Review · doi:10.1017/mor.2023.21 · Cited by 128 · see citing works
“Another line of research by Dietvorst, Simmons, and Massey (2018) has found that people are less averse to algorithms if they have the power to modify how algorithms work.”
BackgroundAn Integrative Perspective on Algorithm Aversion and Appreciation in Decision-Making
Jussupow, E., Benbasat, I., & Heinzl, A. (2024) · MIS Quarterly · doi:10.25300/misq/2024/18512 · Cited by 131 · see citing works
- Authors
- Dietvorst, B. J., Simmons, J. P., & Massey, C.
- Affiliations
- University of Chicago (US)
- University of Pennsylvania (US)
- Venue
- Management Science
- Published
- 2018-03
- Volume
- 64
- Issue
- 3
- Pages
- 1155-1170
- ISSN
- 0025-1909 (ISSN-L)
- 1526-5501
- OpenAlex
- W3121705224
- Semantic Scholar
- e1706933e9e1d15515c3a444454c15890e6d134f
- Cited by
- 1,230
- References
- 27
- Citations by year
- 2026: 260 · 2025: 218 · 2024: 203 · 2023: 179 · 2022: 138 · 2021: 104 · 2020: 58 · 2019: 46 · 2018: 16 · 2017: 6 · 2016: 2
- FWCI
- 20.17
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Locations
- Management Science (publishedVersion)
- SSRN Electronic Journal (acceptedVersion)
- ScholarlyCommons (University of Pennsylvania) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Keywords
- algorithm aversion, forecasting, trust in algorithms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
