The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
The relative influence of advice from human experts and statistical methods on forecast adjustments
Open accessAbstract
Decision makers and forecasters often receive advice from different sources including human experts and statistical methods. This research examines, in the context of stock price forecasting, how the apparent source of the advice affects the attention that is paid to it when the mode of delivery of the advice is identical for both sources. In Study 1, two groups of participants were given the same advised point and interval forecasts. One group was told that these were the advice of a human expert and the other that they were generated by a statistical forecasting method. The participants were then asked to adjust forecasts they had previously made in light of this advice. While in both cases the advice led to improved point forecast accuracy and better calibration of the prediction intervals, the advice which apparently emanated from a statistical method was discounted much more severely. In Study 2, participants were provided with advice from two sources. When the participants were told that both sources were either human experts or both were statistical methods, the apparent statistical‐based advice had the same influence on the adjusted estimates as the advice that appeared to come from a human expert. However when the apparent sources of advice were different, much greater attention was paid to the advice that apparently came from a human expert. Theories of advice utilization are used to identify why the advice of a human expert is likely to be preferred to advice from a statistical method. Copyright © 2009 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“more strongly weigh human input than algorithmic input (Önkal, Goodwin, Thomson, Gönül, & Pollock, 2009; Promberger & Baron, 2006)”
From Algorithm aversion: People erroneously avoid algorithms after seeing them err. (2015) · cites this paper as Önkal, Goodwin, Thomson, Gönül, & Pollock, 2009
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Agency Theory: An Assessment and Review
Eisenhardt, K. M. (1989) · The Academy of Management Review · doi:10.5465/amr.1989.4279003 · Cited by 13,338
“Clinical versus Actuarial Judgment.”
Dawes, R. M., Faust, D., & Meehl, P. E. (1989) · Science · doi:10.1126/science.2648573 · Cited by 2,298
Clinical versus statistical prediction: A theoretical analysis and a review of the evidence.
Meehl, P. E. (1954) · University of Minnesota Press eBooks · doi:10.1037/11281-000 · Cited by 2,171
Grove, W. M., & Meehl, P. E. (1996) · Psychology, Public Policy, and Law · doi:10.1037/1076-8971.2.2.293 · Cited by 1,457
Expert political judgment: how good is it? how can we know?
(2006) · Choice Reviews Online · doi:10.5860/choice.43-3634 · Cited by 1,260
The dominance analysis approach for comparing predictors in multiple regression.
Azen, R., & Budescu, D. V. (2003) · Psychological Methods · doi:10.1037/1082-989x.8.2.129 · Cited by 1,086
Bonaccio, S., & Dalal, R. S. (2006) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2006.07.001 · Cited by 1,011
Kruger, J. (1999) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.77.2.221 · Cited by 898
Cited by
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Resistance to Medical Artificial Intelligence
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019) · Journal of Consumer Research · doi:10.1093/jcr/ucz013 · Cited by 2,015
Task-Dependent Algorithm Aversion
Castelo, N., Bos, M. W., Bos, M. W., Lehmann, D. R., & Lehmann, D. R. (2019) · Springer eBooks · doi:10.1007/978-3-031-88067-4_21 · Cited by 1,441
Forecasting: theory and practice
Petropoulos, F., Apiletti, D., Assimakopoulos, V., Babai, M. Z., Barrow, D. K., Ben Taieb, S., Bergmeir, C., Bessa, R. J., Bijak, J., Boylan, J. E., et al. (2022) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.11.001 · Cited by 921
Self-citationA systematic review of algorithm aversion in augmented decision making
Burton, J. W., Stein, M., & Jensen, T. B. (2020) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2155 · Cited by 814
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691
“Various research suggests that users—again, primarily experts— may in certain cases systematically over-rely on the outputs of intelligent systems [26, 19], while in others they may systematically distrust systems, despite reassurances as to their accuracy [57, 24].”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728
What influences algorithmic decision-making? A systematic literature review on algorithm aversion
Mahmud, H., Islam, A. K. M. N., Ahmed, S. I., & Smolander, K. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.121390 · Cited by 601
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527
Artificial Intelligence Coaches for Sales Agents: Caveats and Solutions
Luo, X., Qin, M. S., Fang, Z., & Qu, Z. (2021) · Journal of Marketing · doi:10.1177/0022242920956676 · Cited by 328
Understanding algorithm aversion: When is advice from automation discounted?
Prahl, A., & Van Swol, L. (2017) · Journal of Forecasting · doi:10.1002/for.2464 · Cited by 308
Cheng, X., Bao, Y., Zarifis, A., Gong, W., & Mou, J. (2022) · Internet Research · doi:10.1108/intr-08-2020-0460 · Cited by 300
Asadabadi, M. R., Chang, E., & Saberi, M. (2019) · Cogent Engineering · doi:10.1080/23311916.2019.1623153 · Cited by 283
Human-machine collaboration in managerial decision making
Haesevoets, T., De Cremer, D., Dierckx, K., & Van Hiel, A. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106730 · Cited by 280
Man versus machine: Complex estimates and auditor reliance on artificial intelligence
COMMERFORD, B. P., DENNIS, S. A., JOE, J. R., & ULLA, J. W. (2022) · Journal of Accounting Research · doi:10.1111/1475-679x.12407 · Cited by 270
Keding, C., & Meissner, P. (2021) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.120970 · Cited by 168
'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions
(2018) · doi:10.31235/osf.io/9wqxr · Cited by 161
Kim, J. H., Kim, J., Kim, C., & Kim, S. (. (2023) · Journal of Travel & Tourism Marketing · doi:10.1080/10548408.2023.2293006 · Cited by 144
Trust in artificial intelligence for medical diagnoses
(2020) · Progress in Brain Research · doi:10.1016/bs.pbr.2020.06.006 · Cited by 116
Chugunova, M., & Sele, D. (2022) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2022.101897 · Cited by 118
Meng, L. (., Bie, Y., Yang, M., & Wang, Y. (2025) · Tourism Management · doi:10.1016/j.tourman.2024.104978 · Cited by 123
Human Decision Making with Machine Assistance: An Experiment on Bailing and Jailing
Grgić-Hlača, N., Engel, C., & Gummadi, K. P. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359280 · Cited by 113
What People Want From Their Professionals: Attitudes Toward Decision‐making Strategies
Eastwood, J., Snook, B., & Luther, K. (2012) · Journal of Behavioral Decision Making · doi:10.1002/bdm.741 · Cited by 97
Is artificial intelligence more persuasive than humans? A meta-analysis
Huang, G., & Wang, S. (2023) · Journal of Communication · doi:10.1093/joc/jqad024 · Cited by 95
Effective human–AI work design for collaborative decision-making
Jain, R., Garg, N., & Khera, S. N. (2023) · Kybernetes · doi:10.1108/k-04-2022-0548 · Cited by 85
A meta-analysis of the weight of advice in decision-making
Bailey, P. E., Leon, T., Ebner, N. C., Moustafa, A. A., & Weidemann, G. (2023) · Current Psychology · doi:10.1007/s12144-022-03573-2 · Cited by 82
Do Humans Trust Advice More if it Comes from AI?: An Analysis of Human-AI Interactions
Vodrahalli, K., Daneshjou, R., Gerstenberg, T., & Zou, J. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534150 · Cited by 63
Cabitza, F., Campagner, A., Angius, R., Natali, C., & Reverberi, C. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581095 · Cited by 51
Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making
Suresh, H., Lao, N., & Liccardi, I. (2020) · 12th ACM Conference on Web Science · doi:10.1145/3394231.3397922 · Cited by 49
Antecedents and effects of trust in forecasting advice
Goodwin, P., Sinan Gönül, M., & Önkal, D. (2013) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2012.08.001 · Cited by 46
Self-citationUsing judgment to select and adjust forecasts from statistical models
De Baets, S., & Harvey, N. (2020) · European Journal of Operational Research · doi:10.1016/j.ejor.2020.01.028 · Cited by 44
Gelbrich, K., Roschk, H., Miederer, S., & Kerath, A. (2026) · Journal of Marketing · doi:10.1177/00222429251344139 · Cited by 38
Kaufmann, E. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100028 · Cited by 36
Quantifying machine influence over human forecasters
Abeliuk, A., Benjamin, D. M., Morstatter, F., & Galstyan, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-72690-4 · Cited by 18
“Forecasters put more weight on advice from human experts than statistical models, and discount models more when the same advice is labeled as a statistical prediction 25 .”
BackgroundRosbach, E., Ammeling, J., Krügel, S., Kießig, A., Fritz, A., Ganz, J., Puget, C., Donovan, T., Klang, A., Köller, M. C., et al. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713319 · Cited by 18
Behavioral challenges to professional skepticism in auditors’ data analytics journey
Li, X. (2022) · Maandblad voor Accountancy en Bedrijfseconomie · doi:10.5117/mab.96.78525 · Cited by 11
Do Humans Trust Advice More if it Comes from AI? An Analysis of Human-AI Interactions
Vodrahalli, K., Daneshjou, R., Gerstenberg, T., & Zou, J. (2021) · arXiv · doi:10.48550/arxiv.2107.07015 · Cited by 16
Improving Advisor Systems: Algorithms that Explain Themselves
Neumann, M., & Kuncel, N. R. (2025) · Journal of Business and Psychology · doi:10.1007/s10869-025-10014-x · Cited by 2
Metadata
- Authors
- Önkal, D., Goodwin, P., Thomson, M., Gönül, S., & Pollock, A.
- Venue
- Journal of Behavioral Decision Making
- Publisher
- Wiley
- Published
- 2009-02-09
- Volume
- 22
- Issue
- 4
- Pages
- 390-409
- ISSN
- 0894-3257 (ISSN-L)
- 1099-0771
- OpenAlex
- W2158570360
- Cited by
- 341
- References
- 48
- FWCI
- 6.31
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.