Explanations From Intelligent Systems: Theoretical Foundations and Implications for Practice1
The DOI matches its Crossref record; 120 of the 122 references it cites were found in a registry or an index.
Abstract
Information systems with an "intelligent" or "knowledge" component are now prevalent and include knowledge-based systems, decision support systems, intelligent agents, and knowledge management systems. These systems are in principle capable of explaining their reasoning or justifying their behavior. There appears to be a lack of understanding, however, of the benefits that can flow from explanation use, and how an explanation function should be constructed. Work with newer types of intelligent systems and help functions for everyday systems, such as word-processors, appears in many cases to neglect lessons learned in the past. This paper attempts to rectify this situation by drawing together the considerable body of work on the nature and use of explanations. Empirical studies, mainly with knowledge-based systems, are reviewed and linked to a sound theoretical base. The theoretical base combines a cognitive effort perspective, cognitive learning theory, and Toulmin's model of argumentation. Conclusions drawn from the review have both practical and theoretical significance. Explanations are important to users in a number of circumstances— when the user perceives an anomaly, when they want to learn, or when they need a specific piece of knowledge to participate properly in problem solving. Explanations, when suitably designed, have been shown to improve performance and learning and result in more positive user perceptions of a system. The design is important, however, because it appears that explanations will not be used if the user has to exert "too much" effort to get them. Explanations should be provided automatically if this can be done relatively unobtrusively, or by hypertext links, and should be context-specific rather than generic. Explanations that conform to Toulmin's model of argumentation, in that they provide adequate justification for the knowledge offered, should be more persuasive and lead to greater trust, agreement, satisfaction, and acceptance—of the explanation and possibly also of the system as a whole.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“While knowledge-based systems can provide clear explanations of the reasoning behind their outputs (Wick & Slagle, 1989; Gregor & Benbasat, 1999), models trained by machine learning algorithms may not (Lipton, 2015).”
From Algorithmic Accountability and Public Reason (2018) · cites this paper as Gregor & Benbasat, 1999
In the citing paper
“Receivers may require explanations about how the AI software came to a decision, especially when they must take action based on that decision (Gregor and Benbasat 1999; Watson et al. 2019).”
From How to Design AI for Social Good: Seven Essential Factors (2020) · cites this paper as Gregor and Benbasat 1999
In the citing paper
“Empirical studies (Gregor and Benbasat 1999) suggest that the suitability of an explanation differs among receivers according to their expertise.”
From How to Design AI for Social Good: Seven Essential Factors (2020) · cites this paper as Gregor and Benbasat 1999
Citations
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
User Acceptance of Computer Technology: A Comparison of Two Theoretical Models
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989) · Management Science · doi:10.1287/mnsc.35.8.982 · Cited by 25,709
A BEHAVIORAL MODEL OF RATIONAL CHOICE
Simon, H. A. (1955) · The Quarterly Journal of Economics · doi:10.2307/1884852 · Cited by 15,877
Moore, G. C., & Benbasat, I. (1991) · Information Systems Research · doi:10.1287/isre.2.3.192 · Cited by 9,150
Self-citationHuman agency in social cognitive theory.
Bandura, A. (1989) · American Psychologist · doi:10.1037/0003-066x.44.9.1175 · Cited by 8,263
Toulmin, S. E. (2003) · doi:10.1017/cbo9780511840005 · Cited by 7,101
Axten, N., Newell, A., & Simon, H. A. (1973) · Contemporary Sociology · doi:10.2307/2063712 · Cited by 6,834
Anderson, J. R. (2013) · doi:10.4324/9781315799438 · Cited by 6,761
Newell, A. (1972) · Cited by 6,222
Rational choice and the structure of the environment.
Simon, H. A. (1956) · Psychological Review · doi:10.1037/h0042769 · Cited by 5,919
Research works printed but not linked: 26
- Anderson, 1985, Cognitive Psychology and Its Implications
- Ausubel, 1985, Learning as Constructing Meaning, 71
- Best, 1989, Cognitive Psychology
- Bruner, 1956, A Study of Thinking
- Brunswik, 1952, The Conceptual Foundation of Psychology
- Buchanan, 1984, Expert Systems
- Eining, 1991, The Impact of Expert System Usage on Experiential Learning in an Auditing Setting, Journal of Information Systems, 5, 1
- 1995, Expert Systems with Applications, 8
- Fitts, 1976, Human Performance
- Flowler, 1985, The Relationship Between Cognitive Style and Dialogue Style
- Gregor, S. D. Explanations from Knowledgebased Systems for Human Learning and Problem Solving, unpublished doctoral dissertation, University of Queensland, Brisbane, Australia, 1996a.
- Gregor, 1997, Explanations from Knowledgebased Systems and the Learning versus Working Argument
- Gregor, 1997, Knowledge-based Systems in Prescriptive versus Supportive Roles and the Use of Explanations
- Kidd, 1985, The Consultative Role of an Expert System, 228
- Koonce, 1992, Explanation and Counterexplanation During Audit Analytical Review, The Accounting Review, 67, 59
- Lerch, 1993, Measuring Trust in Machine Advice
- 1981, The Macquarie Dictionary
- Mao, 1996, The Effects of Hypertext-based Explanations in Knowledgebased Systems on Explanation Use and Knowledge Transfer
- Mao, 1996, Exploring the Use of Explanations in Knowledge-based Systems: A Process Tracing Analysis
- Murphy, 1990, Expert System Use and the Development of Expertise in Auditing: A Preliminary Investigation, Journal of Information Systems, 18
- Oltman, 1971, Group Embedded Figures Test
- Schank, 1986, Explanation: A First Pass, 139
- Schroder, 1967, Human Information Processing
- Seligman, 1990, Learned Optimism
- Wognum, P. M. Explanation of Automated Reasoning: How and Why?unpublished doctoral dissertation, University of Twente, Enschede, The Netherlands, 1990.
- Yourdon, 1989, Modern Structure Analysis
Cited by
The Nature of Theory in Information Systems1
Gregor, S. (2006) · MIS Quarterly · doi:10.2307/25148742 · Cited by 3,306
“…(see, for example, Achinstein 1983; Craik 1943; Hempel and Oppenheim 1948; Nagel 1979), in relation to everyday reasoning and the nature of argumentation (see Toulmin 1958; Toulmin et al. 1979), as well as in the subbranches of IS relating to knowledge-based systems (see Gregor and Benbasat 1999).”
BackgroundSelf-citationSynthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987
E-Commerce Product Recommendation Agents: Use, Characteristics, and Impact1
Xiao, B., & Benbasat, I. (2007) · MIS Quarterly · doi:10.2307/25148784 · Cited by 1,081
“…to involve actual use of an RA (prototype or operational, web-based or stand-alone) by human users (similar to the definition of empirical studies in Gregor and Benbasat 1999) in either online or physical settings.8 • The study must have dependent variables that go beyond “accuracy” and…”
BackgroundInfluentialSelf-citationQuestioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
“n kind of explanation...to satisfy certain user purposes of user goals. ” In the early generation of AI work, question-driven frameworks were used to generate explanations for knowledge-based systems [20,38,88]. AQUA [72] is a reasoning model that uses questions to generate explanations and identify knowledge gaps for…”
MethodManaging artificial intelligence
Berente, N., Gu, B., Recker, J., & Santhanam, R. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16274 · Cited by 898
Trust In and Adoption of Online Recommendation Agents
University of British Columbia, C., Benbasat, I., Wang, W., & University of British Columbia, C. (2005) · Journal of the Association for Information Systems · doi:10.17705/1jais.00065 · Cited by 841
“By virtue of making the performance of a system transparent to users, they influence user trust in and acceptance of the systems (Gregor and Benbasat, 1999; Hayes-Roth and Jacobstein, 1994).”
BackgroundInfluentialSelf-citationMittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794
“Gregor and Benbasat (1999) support the importance of abnormality, suggesting that users request explanations when an anomaly or abnormal event is detected.”
BackgroundA Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785
“lack-box machine learning systems in the big-data-era has driven much attention from multiple different communities, interpretability of intelligent systems has also been studied in numerous contexts [96, 38]. The study of personalized agents, recommendation systems, and critical decision-making tasks (e.g., medical a…”
BackgroundA Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11839 · Cited by 783
“While the increasing impact of advanced black-box machine learning systems in the big-data-era has driven much attention from multiple different communities, interpretability of intelligent systems has also been studied in numerous contexts [96, 38].”
BackgroundAbdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
“This is the biggest, most central, and mature subnetwork spanning many research areas of the Intelligent Systems and Ambient Intelligence communities.”
BackgroundArtificial intelligence and knowledge sharing: Contributing factors to organizational performance
Olan, F., Ogiemwonyi Arakpogun, E., Suklan, J., Nakpodia, F., Damij, N., & Jayawickrama, U. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2022.03.008 · Cited by 542
“IS perception processes consist of acquisition, abstractions, and filtering of data before transporting it for the next action (Gregor & Benbasat, 1999).”
BackgroundSchultze, U., & Leidner, D. E. (2002) · MIS Quarterly · doi:10.2307/4132331 · Cited by 667
“There are papers that examine explanations in knowledgebased systems (Dhaliwal and Benbasat 1996; Gregor and Benbasat 1999), as well as papers that deal with knowledge representation (Lee and O'Keefe 1996; Jonas and Laios 1993; Nissen 2000).”
BackgroundLanger, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & Baum, K. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103473 · Cited by 652
“At the same time, when novice users want to learn how to use a system, they might want detailed explanatory information whereas when expert users want to use the system for task fulfillment, every unnecessary piece of information could lead to the rejection of the system [36].”
BackgroundInfluentialNotions of explainability and evaluation approaches for explainable artificial intelligence
Vilone, G., & Longo, L. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.009 · Cited by 598
Workman, M., Bommer, W. H., & Straub, D. (2008) · Computers in Human Behavior · doi:10.1016/j.chb.2008.04.005 · Cited by 554
“One aspect of this is seen in studies (e.g., Adams, Nelson, & Todd, 1992, Gregor & Benbasat, 1999) that show how technology acceptance and ease-of-use (Davis, Bagozzi, & Warshaw, 1989) impacts assessments about whether or not to use a given technology.”
BackgroundWhy and why not explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552
“In KBS, several expert systems (e.g., MYCIN [9]) provided reasoning trace explanations to support Why, Why Not and How To questions, but the comparative benefits of these were not compared.”
BackgroundInfluentialThe Ethics of Artificial Intelligence: Principles, Challenges, and Opportunities
Floridi, L. (2023) · doi:10.1093/oso/9780198883098.001.0001 · Cited by 507
Recommendation Agents for Electronic Commerce: Effects of Explanation Facilities on Trusting Beliefs
Wang, W., & Benbasat, I. (2007) · Journal of Management Information Systems · doi:10.2753/mis0742-1222230410 · Cited by 498
“Various KBS explanations and different ways to classify them are summarized in the literature [19, 28].”
BackgroundMethodInfluentialSelf-citationAlgorithmic Accountability and Public Reason
Binns, R. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0263-5 · Cited by 457
“…systems can 9 Although, significant challenges in specifying constraints remain; see e.g. (Bostrom, 2003). provide clear explanations of the reasoning behind their outputs (Wick & Slagle, 1989; Gregor & Benbasat, 1999), models trained by machine learning algorithms may not (Lipton, 2015).”
BackgroundThe effects of transparency on trust in and acceptance of a content-based art recommender
Cramer, H., Evers, V., Ramlal, S., van Someren, M., Rutledge, L., Stash, N., Aroyo, L., & Wielinga, B. (2008) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-008-9051-3 · Cited by 448
“Other prior studies also suggest that making adaptive systems more transparent to the user could lead to a more positive user attitude towards using a system and increases in system performance (e.g. Kay 2006; Höök et al. 1996; Höök 2000; Carmagnola et al. 2006; Gregor and Benbasat 1999).”
BackgroundA systematic review and taxonomy of explanations in decision support and recommender systems
Nunes, I., & Jannach, D. (2017) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-017-9195-0 · Cited by 395
“A number of explanation taxonomies of different granularity levels has been proposed in the literature in the past (Ye and Johnson 1995; Chandrasekaran et al. 1989; Gregor and Benbasat 1999; Lacave and Díez 2002, 2004; Nakatsu 2006; Papadimitriou et al. 2012; Vig et al. 2009; Blanco et al. 2012; Friedrich and Zanker 2…”
BackgroundThe Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441
How to Design AI for Social Good: Seven Essential Factors
Floridi, L., Cowls, J., King, T. C., & Taddeo, M. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00213-5 · Cited by 430
“Empirical studies (Gregor and Benbasat 1999) suggest that the suitability of an explanation differs among receivers according to their expertise.”
BackgroundKnowledge Management and Organizational Learning
KING, W., CHUNG, T., & HANEY, M. (2008) · Omega · doi:10.1016/j.omega.2006.07.004 · Cited by 391
Explanations as Mechanisms for Supporting Algorithmic Transparency
Rader, E., Cotter, K., & Cho, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173677 · Cited by 388
“Explanations present information about how the system produced the recommendation and the reasons behind it [3, 24, 52], and are in service to the system’s purpose and goals [20].”
BackgroundExplainable artificial intelligence: Objectives, stakeholders, and future research opportunities
Meske, C., Bunde, E., Schneider, J., & Gersch, M. (2022) · Information Systems Management · doi:10.1080/10580530.2020.1849465 · Cited by 365
“…was derived), type III, control or strategic (which explain the system’s control behavior and problem-solving strategy), and type IV, terminological (which supply definitional or terminological information) (Gregor & Benbasat, 1999, based on Chandrasekaran et al., 1989; Swartout & Smoliar, 1987).”
BackgroundInfluential“The reports of my death are greatly exaggerated”—Artificial intelligence research in accounting
Sutton, S. G., Holt, M., & Arnold, V. (2016) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2016.07.005 · Cited by 245
Credibility: A multidisciplinary framework
Rieh, S. Y., & Danielson, D. R. (2007) · Annual Review of Information Science and Technology · doi:10.1002/aris.2007.1440410114 · Cited by 336
“Finally, terminological explanations provide definition information (Gregor & Benbasat, 1999; Ye & Johnson, 1995).”
InfluentialExpl(AI)ned: The Impact of Explainable Artificial Intelligence on Users’ Information Processing
Bauer, K., von Zahn, M., & Hinz, O. (2023) · Information Systems Research · doi:10.1287/isre.2023.1199 · Cited by 261
“…et al. 2015), being able to modify predictions (Dietvorst et al. 2018), the divergence bet-ween actual and expected predictive performance (Jus-supow et al. 2020), and, most importantly for our research, understanding the system’s internal logic (Gregor and Benbasat 1999, Hemmer et al. 2021).”
ResultSocial Bias Frames: Reasoning about Social and Power Implications of Language
Sap, M., Gabriel, S., Qin, L., Jurafsky, D., Smith, N. A., & Choi, Y. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.486 · Cited by 332
Designing transparency for effective Human-AI collaboration
Vössing, M., Kühl, N., Lind, M., & Satzger, G. (2022) · Information Systems Frontiers · doi:10.1007/s10796-022-10284-3 · Cited by 223
“Research shows that explanations can help establish trust (Nunes and Jannach, 2017; Gregor & Benbasat, 1999).”
BackgroundUtilizing big data analytics for information systems research: challenges, promises and guidelines
Müller, O., Junglas, I., Brocke, J. V., & Debortoli, S. (2016) · European Journal of Information Systems · doi:10.1057/ejis.2016.2 · Cited by 281
Knowledge graphs as tools for explainable machine learning: A survey
Tiddi, I., & Schlobach, S. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2021.103627 · Cited by 283
Kocielnik, R., Amershi, S., & Bennett, P. N. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300641 · Cited by 277
“In most cases, however, transparency techniques are intended for post fact explanations about the decisions of an AI or ML system [19, 22], rather than for adjusting user expectations of an AI-powered system prior to its use.”
BackgroundExplainable Artificial Intelligence: a Systematic Review
Vilone, G., & Longo, L. (2020) · arXiv · doi:10.48550/arxiv.2006.00093 · Cited by 269
“Other scholars instead focused on reviewing methods for knowledge-based approaches such as Expert Systems (ES) [32] and Intelligent Systems [33].”
BackgroundExplaining Data-Driven Document Classifications1
Martens, D., & Provost, F. (2014) · MIS Quarterly · doi:10.25300/misq/2014/38.1.04 · Cited by 264
“As pointed out by Gregor and Benbasat (1999), a user will want an explanation when she perceives an anomaly.”
BackgroundInfluentialThe perils and pitfalls of explainable AI: Strategies for explaining algorithmic decision-making
de Bruijn, H., Warnier, M., & Janssen, M. (2022) · Government Information Quarterly · doi:10.1016/j.giq.2021.101666 · Cited by 264
Explainability in human–agent systems
Rosenfeld, A., & Richardson, A. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09408-y · Cited by 224
Bringing Transparency Design into Practice
Eiband, M., Schneider, H., Bilandzic, M., Fazekas-Con, J., Haug, M., & Hussmann, H. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172961 · Cited by 250
“Similarly, Schaffer et al. [40] as well as Gregor and Benbasat [13] highlight the importance of considering the user group or individual differences between users when thinking about what to explain : Explanations turned out to be more useful when tailored to the interest of a specific user group, and may even reduce u…”
BackgroundResultInfluentialAssessing demand for intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2009) · Proceedings of the 11th international conference on Ubiquitous computing · doi:10.1145/1620545.1620576 · Cited by 246
“Gregor and Benbasat’s [16] meta-review investigates explanation types that users of knowledge-based systems (KBS) would like to have.”
BackgroundWang, W., & Benbasat, I. (2008) · Journal of Management Information Systems · doi:10.2753/mis0742-1222240410 · Cited by 241
“Explanations are important components of decision support technologies, including RAs, because by making the performance of technologies transparent to their users, they improve user trust in the systems [20, 34, 67].”
BackgroundSelf-citationExplanation in Case-Based Reasoning—Perspectives and Goals
Sørmo, F., Cassens, J., & Aamodt, A. (2005) · Artificial Intelligence Review · doi:10.1007/s10462-005-4607-7 · Cited by 239
“As such, this goal also contains the category of justification explanations from Gregor and Benbasat (1999).”
BackgroundMethodInfluentialKim, D., & Benbasat, I. (2006) · Information Systems Research · doi:10.1287/isre.1060.0093 · Cited by 239
Self-citationRequirements-Aware Systems: A Research Agenda for RE for Self-adaptive Systems
Sawyer, P., Bencomo, N., Whittle, J., Letier, E., & Finkelstein, A. (2010) · 2010 18th IEEE International Requirements Engineering Conference · doi:10.1109/re.2010.21 · Cited by 224
“The provision of understandable explanations of system behaviour has been studied in a number of research fields: principally, in knowledge-based systems [34][37], intelligent tutoring systems [31], context-aware systems [36][33], and debugging [38].”
Vorm, E. S., & Combs, D. J. Y. (2022) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2070107 · Cited by 197
“Studies in information systems have defined it as “… explaining to their human users both the knowledge they contain and the reasoning processes they go through’’ (Gregor & Benbasat, 1999, p. 498).”
BackgroundWang, W., & Benbasat, I. (2009) · MIS Quarterly · doi:10.2307/20650293 · Cited by 225
Self-citationAsatiani, A., University of Gothenburg, S., Malo, P., Aalto University School of Business, F., Nagbøl, P. R., IT University of Copenhagen, D., Penttinen, E., Aalto University School of Business, F., Rinta-Kahila, T., The University of Queensland, A., et al. (2021) · Journal of the Association for Information Systems · doi:10.17705/1jais.00664 · Cited by 173
How Knowledge Validation Processes Affect Knowledge Contribution
Durcikova, A., & Gray, P. (2009) · Journal of Management Information Systems · doi:10.2753/mis0742-1222250403 · Cited by 182
“…and may also affect their contribution behavior. research has also suggested that experts judge knowledge content differently than do novices [36, 82]; thus, expertise may play a role in predicting both perceived knowledge quality and contribution frequency. figure 1 summarizes the research…”
BackgroundSong, X., Xu, B., & Zhao, Z. (2022) · Information & Management · doi:10.1016/j.im.2022.103595 · Cited by 192
How to Design AI for Social Good: Seven Essential Factors
Floridi, L., Cowls, J., King, T. C., & Taddeo, M. (2021) · Philosophical Studies Series · doi:10.1007/978-3-030-81907-1_9 · Cited by 161
Interacting with Recommenders—Overview and Research Directions
Jugovac, M., & Jannach, D. (2017) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3001837 · Cited by 172
Langer, M., & König, C. J. (2021) · Human Resource Management Review · doi:10.1016/j.hrmr.2021.100881 · Cited by 196
“If users want to quickly fulfill daily tasks and routines, any additional information that accompanies system outputs might lead to negative reactions and decreasing efficiency (Gregor & Benbasat, 1999; Tintarev & Masthoff, 2007).”
InfluentialKowatsch, T., & Maass, W. (2010) · Computers in Human Behavior · doi:10.1016/j.chb.2010.01.006 · Cited by 163
“Correspondingly, mobile recommendation agents (MRAs) may also increase the value of product information in bricks-and-mortar stores.”
BackgroundSocial network analysis for customer churn prediction
Verbeke, W., Martens, D., & Baesens, B. (2014) · Applied Soft Computing · doi:10.1016/j.asoc.2013.09.017 · Cited by 164
Dowling, C., & Leech, S. A. (2012) · Contemporary Accounting Research · doi:10.1111/1911-3846.12010 · Cited by 201
Wang, W., & Benbasat, I. (2016) · Journal of Management Information Systems · doi:10.1080/07421222.2016.1243949 · Cited by 145
“Users of a transparent RA have a clear mental model of the RA and will be able to make these assessments [17].”
BackgroundInfluentialSelf-citationArnold, V., Clark, N., Collier, P. A., Leech, S. A., & Sutton, S. G. (2006) · MIS Quarterly · doi:10.2307/25148718 · Cited by 166
Manning, L., Brewer, S., Craigon, P. J., Frey, J. G., Gutiérrez, A., Jacobs, N. J., Kanza, S., Munday, S., Sacks, J., & Pearson, S. (2022) · Trends in Food Science & Technology · doi:10.1016/j.tifs.2022.04.025 · Cited by 184
“…has been linked with either being intelligent, being knowledge-based, providing meaning, creating understanding, reconciling differences (Gregor & Benbasat, 1999); or a process of communication and interpretation, “ facilitating the human user ’ s understanding of the agent ’ s logic ”…”
BackgroundThe communal resource and information systems
von Krogh, G. (2002) · The Journal of Strategic Information Systems · doi:10.1016/s0963-8687(02)00006-9 · Cited by 163
Westphal, M., Vössing, M., Satzger, G., Yom‐Tov, G. B., & Rafaeli, A. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2023.107714 · Cited by 180
How Much Automation Is Too Much? Keeping the Human Relevant in Knowledge Work
Sutton, S. G., Arnold, V., & Holt, M. (2018) · Journal of Emerging Technologies in Accounting · doi:10.2308/jeta-52311 · Cited by 123
Explanation and trust: what to tell the user in security and AI?
Pieters, W. (2011) · Ethics and Information Technology · doi:10.1007/s10676-010-9253-3 · Cited by 153
“Benefits of explanations in intelligent systems are discussed by Gregor and Benbasat ( 1999 ) .”
Background(2000) · Journal of Management Information Systems · doi:10.1080/07421222.2000.11045646 · Cited by 143
Multi-criteria decision support and evaluation of strategies for nuclear remediation management☆
Geldermann, J., Bertsch, V., Treitz, M., French, S., Papamichail, K. N., & Hämäläinen, R. P. (2007) · Omega · doi:10.1016/j.omega.2006.11.006 · Cited by 144
“Explanation facilities contribute to positive user attitudes and improve user performance [30].”
BackgroundToolkit to support intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2010) · Proceedings of the 12th ACM international conference on Ubiquitous computing · doi:10.1145/1864349.1864353 · Cited by 143
“, [17, 23, 26, 27]) and specifically, Lim & Dey [25] describe a set of ten explanation types that context-aware applications should provide for end-users.”
BackgroundDesign principles for a hybrid intelligence decision support system for business model validation
Dellermann, D., Lipusch, N., Ebel, P., & Leimeister, J. M. (2018) · Electronic Markets · doi:10.1007/s12525-018-0309-2 · Cited by 168
“Therefore, combining different formats of presenting the results are needed, such as text-based and multi-media (Gregor and Benbasat 1999).”
BackgroundInfluentialProviding Decisional Guidance for Multicriteria Decision Making in Groups
Limayem, M., & DeSanctis, G. (2000) · Information Systems Research · doi:10.1287/isre.11.4.386.11874 · Cited by 243
“Within Gregor and Benbasat’s (1999) classification scheme, the explanations were supportive in their content (since reasoning was justified by linking the explanation to knowledge from which it was derived), multimedia in presentation format (since tables and graphs were provided in addition to text), and included bot…”
BackgroundExplaining Data-Driven Decisions made by AI Systems: The Counterfactual Approach
Fernández-Loría, C., Provost, F. J., & Han, X. (2022) · MIS Quarterly · doi:10.25300/misq/2022/16749 · Cited by 165
Arguing the Value of Virtual Worlds: Patterns of Discursive Sensemaking of an Innovative Technology1
Berente, N., Hansen, S., Pike, J. C., & Bateman, P. J. (2011) · MIS Quarterly · doi:10.2307/23042804 · Cited by 137
Li, M., & Mao, J. (2015) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2014.12.004 · Cited by 143
Can we do better explanations? A proposal of user-centered explainable AI
Ribera, M., & Lapedriza, À. (2019) · IUI Workshops · Cited by 133
The design features of forecasting support systems and their effectiveness
Fildes, R. A., Goodwin, P. B., & Lawrence, M. J. (2005) · Decision Support Systems · doi:10.1016/j.dss.2005.01.003 · Cited by 155
“This is because they fail to reveal assumptions or encourage alternative views of the problem (Gregor & Benbasat, 1999).”
BackgroundCollaborating with technology-based autonomous agents: Issues and research opportunities
Seeber, I., Waizenegger, L., Seidel, S., Morana, S., Benbasat, I., & Lowry, P. B. (2020) · Internet Research · doi:10.1108/intr-12-2019-0503 · Cited by 151
Self-citationBrasse, J., Broder, H. R., Förster, M., Klier, M., & Sigler, I. (2023) · Electronic Markets · doi:10.1007/s12525-023-00644-5 · Cited by 149
“…at approaches that make AI systems more explainable—sometimes also referred to as compre-hensible (Doran et al., 2018)—by automatically generating explanations for their functioning and outcomes while maintaining the AI’s high performance levels (Adadi & Berrada, 2018; Gregor & Benbasat, 1999).”
BackgroundI can do better than your AI: expertise and explanations
Schaffer, J., O'Donovan, J., Michaelis, J., Raglin, A., & Höllerer, T. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302308 · Cited by 119
“Research into explanations started with expert systems studies [1, 28], and more recently their trust and persuasion effects have been studied in recommender systems [36, 50].”
BackgroundAbedin, B. (2021) · Internet Research · doi:10.1108/intr-05-2020-0300 · Cited by 130
“Historically, explanations of AI emerged first in the context of rule-based expert systems and were viewed as elements of designing a system capable of producing drill-down outputs (Biran and Cotton, 2017; Gregor and Benbasat, 1999).”
BackgroundEvaluating and understanding text-based stock price prediction models
de Fortuny, E. J., Smedt, T. D., Martens, D., & Daelemans, W. M. P. (2014) · Information Processing & Management · doi:10.1016/j.ipm.2013.12.002 · Cited by 116
“Moreover, prior research has shown that the ability to explain a decision support model is of crucial importance for the acceptance of such models (Gregor & Benbasat, 1999; Kayande, De Bruyn, Lilien, Rangaswamy, & van Bruggen, 2008; Martens & Provost, 2014).”
BackgroundA Dss Classification Model for Research in Human Resource Information Systems
Beckers, A. M., & Bsat, M. Z. (2002) · Information Systems Management · doi:10.1201/1078/43201.19.3.20020601/37169.6 · Cited by 116
Cognition Matters: Enduring Questions in Cognitive IS Research
The University of Melbourne, Davern, M., Shaft, T., The University of Oklahoma, Te'eni, D., & Tel Aviv University (2012) · Journal of the Association for Information Systems · doi:10.17705/1jais.00290 · Cited by 108
“For example, in an Expert Systems (ES) context, Davern and Parkes (2010) find that the taxonomy of types of guidance appears to be incommensurable with a well-established taxonomy of the forms of explanation mechanism in ES (Gregor & Benbasat, 1999).”
BackgroundDigital Responsibility: A Multilevel Framework for Responsible Digitalization
Trier, M., Kundisch, D., Beverungen, D., Müller, O., Schryen, G., Mirbabaie, M., & Trang, S. T.-N. (2023) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00822-x · Cited by 125
“Already more than two decades ago, IS researchers recognized the need for explaining the logic and outputs of intelligent black-box systems to users ( transparency principle ) in order to increase their acceptance and effective use (Gregor and Benbasat 1999).”
BackgroundDesign and evaluation of an intelligent decision support system for nuclear emergencies
Papamichail, K. N., & French, S. (2004) · Decision Support Systems · doi:10.1016/j.dss.2004.04.014 · Cited by 107
“Explanation facilities improve performance and learning [32] and help a range of users including experienced professionals and novices [56].”
BackgroundAre explanations always important?: a study of deployed, low-cost intelligent interactive systems
Bunt, A., Lount, M., & Lauzon, C. (2012) · Proceedings of the 2012 ACM international conference on Intelligent User Interfaces · doi:10.1145/2166966.2166996 · Cited by 106
“According to Gregor and Benbasat s survey of a number of studies in this domain [7], explanations can improve user performance, helping users make faster and better decisions.”
BackgroundInfluentialDineen, B. R., Noe, R. A., & Wang, C. (2004) · Human Resource Management · doi:10.1002/hrm.20011 · Cited by 104
The role of domain expertise in trusting and following explainable AI decision support systems
Bayer, S., Gimpel, H., & Markgraf, M. (2022) · Journal of Decision Systems · doi:10.1080/12460125.2021.1958505 · Cited by 103
“Explanations delivered by AI systems have three dimensions (Gregor & Benbasat, 1999).”
BackgroundMethodInfluentialThe effects of structural characteristics of explanations on use of a DSS
Gonul, M. S., Önkal, D., & Lawrence, M. J. (2006) · Decision Support Systems · doi:10.1016/j.dss.2005.12.003 · Cited by 103
“[3] [10] [34]).”
Kim, D., Song, Y., Kim, S., Lee, S., Wu, Y., Shin, J., & Lee, D. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.122343 · Cited by 110
Knowledge creation in small‐firm network
Balestrin, A., Vargas, L. M., & Fayard, P. (2008) · Journal of Knowledge Management · doi:10.1108/13673270810859541 · Cited by 99
Detecting task demand via an eye tracking machine learning system
Shojaeizadeh, M., Djamasbi, S., Paffenroth, R. C., & Trapp, A. C. (2018) · Decision Support Systems · doi:10.1016/j.dss.2018.10.012 · Cited by 105
“7 Conclusion Because users place a high value on conserving cognitive resources [32,62,74-76], developing computerized tools to help people manage their cognitive resources can help them be more effective in decision making.”
BackgroundFinancial education and digital asset management: What's in the black box?
Litterscheidt, R., & Streich, D. J. (2020) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2020.101573 · Cited by 114
XAIR: A Framework of Explainable AI in Augmented Reality
Xu, X., Yu, A., Jonker, T. R., Todi, K., Lu, F., Qian, X., Belo, J. M. E., Wang, T., Li, M., Mun, A., et al. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581500 · Cited by 109
“With the increasing prevalence of advanced black-box models that make more critical predictions and decisions, the interpretability and transparency of AI systems has attracted increasing attention from various academic, industrial, and regulatory stakeholders [1, 90, 94, 169].”
BackgroundFairness and Transparency in Recommendation: The Users’ Perspective
Sonboli, N., Smith, J. J., Berenfus, F. C., Burke, R., & Fiesler, C. L. (2021) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3450613.3456835 · Cited by 115
“Transparency has become an essential feature of algorithmic design, especially in application areas that are socially sensitive due to (1) legal requirements for transparency such as the GDPR [24, 41]; (2) the need to build users’ trust in a system [11, 29, 38]; (3) the need for error detection to help mitigate bias a…”
BackgroundRamon, Y., Martens, D., Provost, F., & Evgeniou, T. (2020) · Advances in Data Analysis and Classification · doi:10.1007/s11634-020-00418-3 · Cited by 105
“The importance of understanding individual classification decisions is well-argued in the literature (Gregor and Benbasat 1999; Freitas 2014; Martens and Provost 2014; Goodman and Flaxman 2016; Doshi-Velez and Kim 2017; Ras et al. 2018; Lipton 2018).”
BackgroundExpert decision support system use, disuse, and misuse: a study using the theory of planned behavior
Workman, M. (2004) · Computers in Human Behavior · doi:10.1016/j.chb.2004.03.011 · Cited by 86
“Recent advancements in expert systems technology, however, have established a new collaborative interchange (Gregor & Benbasat, 1999).”
BackgroundMethodInfluentialKwon, O., Yoo, K., & Suh, E. (2004) · Expert Systems with Applications · doi:10.1016/j.eswa.2004.08.007 · Cited by 85
Designing AI for Social Good: Seven Essential Factors
Cowls, J., King, T., Taddeo, M., & Floridi, L. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3388669 · Cited by 83
“Empirical studies (Gregor and Benbasat 1999) suggest that the suitability of an explanation differs among receivers according to their expertise.”
BackgroundRegret avoidance as a measure of DSS success: An exploratory study
Hung, S., Ku, Y.-C., Liang, T., & Lee, C.-J. (2006) · Decision Support Systems · doi:10.1016/j.dss.2006.05.006 · Cited by 91
“informative guidance Barr and Sharda [6] DSS Decision performance González and Kasper [29] Animation images Decision quality Animation transitions Animation navigation Swink and Robinson [65] DSS attributes Decision performances Problem size Network types Demand dispersion patterns van Bruggen, Smidts, and Wierenga [7…”
Methodd.tour: style-based exploration of design example galleries
Ritchie, D., Kejriwal, A. A., & Klemmer, S. R. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047216 · Cited by 73
“Explaining recommendations can improve user experience, build trust, and support evaluation of system conclusions [28, 25, 9]. d.tour provides the Ambient Style Display to help explain search results.”
BackgroundImpact of GDSS: Opening the black box
Limayem, M., Banerjee, P. K., & Ma, L. C. K. (2005) · Decision Support Systems · doi:10.1016/j.dss.2005.08.004 · Cited by 82
“An intelligent guidance system monitors users’ behavior and provides cues and customized explanations accordingly [32].”
BackgroundTechnology Dominance in Complex Decision Making: The Case of Aided Credibility Assessment
Jensen, M. L., Lowry, P. B., Burgoon, J. K., & Nunamaker, J. F. (2010) · Journal of Management Information Systems · doi:10.2753/mis0742-1222270108 · Cited by 92
“…how the decision aid arrives at its recommendation in order to determine if the recommendation is suitable for acceptance [2]. gregor and Benbasat [31] suggested that professionals will use explanations to resolve anomalies and to verify the performance of the decision aid. this phenomenon has…”
BackgroundMirror, Mirror on the Wall: Algorithmic Assessments, Transparency, and Self-Fulfilling Prophecies
Bauer, K., & Gill, A. (2024) · Information Systems Research · doi:10.1287/isre.2023.1217 · Cited by 64
“While focusing mainly on system users, not data subjects, previous research on explainable AI (XAI) provides ample evidence that explainability can influence human attitudes toward and perceptions of algorithmic systems (Gregor and Benbasat 1999, Poursabzi-Sangdeh et al. 2021).”
Background
Metadata
- Authors
- Gregor, S., & Benbasat, I.
- Affiliations
- Central Queensland University (AU)
- University of British Columbia (CA)
- Venue
- MIS Quarterly
- Publisher
- MIS Quarterly
- Published
- 1999-12-01
- Volume
- 23
- Issue
- 4
- Pages
- 497-530
- ISSN
- 0276-7783 (ISSN-L)
- 2162-9730
- OpenAlex
- W1576620340
- Cited by
- 535
- References
- 122
- Citations by year
- 2026: 28 · 2025: 31 · 2024: 20 · 2023: 47 · 2022: 22 · 2021: 43 · 2020: 32 · 2019: 30 · 2018: 21 · 2017: 21 · 2016: 13 · 2015: 11 · 2014: 17 · 2013: 6 · 2012: 22
- FWCI
- 7.04
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- MIS Quarterly (publishedVersion)
- Journal of the Association for Information Systems (submittedVersion)
- Funders
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- explanation generation, intelligent systems, expert systems, decision support system, intelligent agents, knowledge management systems, Toulmin model, cognitive effort, cognitive learning theory, explanation design, user trust, user satisfaction, system acceptance, word processing software
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.