How Incorporating Feedback Mechanisms in a DSS Affects DSS Evaluations
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Model-based decision support systems (DSS) improve performance in many contexts that are data-rich, uncertain, and require repetitive decisions. But such DSS are often not designed to help users understand and internalize the underlying factors driving DSS recommendations. Users then feel uncertain about DSS recommendations, leading them to possibly avoid using the system. We argue that a DSS must be designed to induce an alignment of a decision maker's mental model with the decision model embedded in the DSS. Such an alignment requires effort from the decision maker and guidance from the DSS. We experimentally evaluate two DSS design characteristics that facilitate such alignment: (i) feedback on the upside potential for performance improvement and (ii) feedback on corrective actions to improve decisions. We show that, in tandem, these two types of DSS feedback induce decision makers to align their mental models with the decision model, a process we call deep learning, whereas individually these two types of feedback have little effect on deep learning. We also show that deep learning, in turn, improves user evaluations of the DSS. We discuss how our findings could lead to DSS design improvements and better returns on DSS investments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
Information Systems Success: The Quest for the Dependent Variable
DeLone, W. H., & McLean, E. R. (1992) · Information Systems Research · doi:10.1287/isre.3.1.60 · Cited by 10,697
Agresti, A. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/0471249688 · Cited by 6,746
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882
Managerial Influence in the Implementation of New Technology
Leonard-Barton, D., & Deschamps, I. (1988) · Management Science · doi:10.1287/mnsc.34.10.1252 · Cited by 812
Cognitive Fit: An Empirical Study of Information Acquisition
Vessey, I., & Galletta, D. (1991) · Information Systems Research · doi:10.1287/isre.2.1.63 · Cited by 705 · cited 1×
“information format and task ( Vessey and Galletta 1991 ), accessibility (e.g., Mawhinney and Lederer”
Perceived information quality in data exchanges: Effects on risk, trust, and intention to use
Nicolaou, A. I., & McKnight, D. H. (2006) · Information Systems Research · doi:10.1287/isre.1060.0103 · Cited by 682
Evaluating the Impact of DSS, Cognitive Effort, and Incentives on Strategy Selection
Todd, P., & Benbasat, I. (1999) · Information Systems Research · doi:10.1287/isre.10.4.356 · Cited by 312
Feedback Specificity, Exploration, and Learning.
Goodman, J. S., Wood, R. E., & Hendrickx, M. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.2.248 · Cited by 201
Cited by
Mittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794
“…machine learning models, it is perhaps most useful to always treat explanation generation as an interactive process, initially involving a mix of human and automated actors, at a minimum an inquirer (e.g. a developer, user) and the model or system (Kayande et al., 2009; Martens and Provost, 2013).”
BackgroundWhat influences algorithmic decision-making? A systematic literature review on algorithm aversion
Mahmud, H., Islam, A. K. M. N., Ahmed, S. I., & Smolander, K. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.121390 · Cited by 601
The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems
Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., & Ebel, P. (2021) · arXiv · doi:10.48550/arxiv.2105.03354 · Cited by 456
Utilizing big data analytics for information systems research: challenges, promises and guidelines
Müller, O., Junglas, I., Brocke, J. V., & Debortoli, S. (2016) · European Journal of Information Systems · doi:10.1057/ejis.2016.2 · Cited by 281
The Future of Human-AI Collaboration: A Taxonomy of Design Knowledge for Hybrid Intelligence Systems
Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., & Ebel, P. (2019) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2019.034 · Cited by 279
Metadata
- Authors
- Kayande, U., De Bruyn, A., Lilien, G. L., Rangaswamy, A., & van Bruggen, G. H.
- Venue
- Information Systems Research
- Published
- 2009-12
- Volume
- 20
- Issue
- 4
- Pages
- 527-546
- ISSN
- 1047-7047 (ISSN-L)
- 1526-5536
- OpenAlex
- W2104454321
- Cited by
- 116
- References
- 9
- FWCI
- 5.15
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How Incorporating Feedback Mechanisms in a DSS Affects DSS Evaluations. https://app.cochat.ai/papers/10.1287/isre.1080.0198
- The paper
- Kayande, U., De Bruyn, A., Lilien, G. L., Rangaswamy, A., & van Bruggen, G. H. (2009). How Incorporating Feedback Mechanisms in a DSS Affects DSS Evaluations. Information Systems Research. https://doi.org/10.1287/isre.1080.0198
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.