Explanations From Intelligent Systems: Theoretical Foundations and Implications for Practice1
The DOI matches its Crossref record; 120 of the 122 references it cites were found in a registry or an index.
Abstract
Information systems with an "intelligent" or "knowledge" component are now prevalent and include knowledge-based systems, decision support systems, intelligent agents, and knowledge management systems. These systems are in principle capable of explaining their reasoning or justifying their behavior. There appears to be a lack of understanding, however, of the benefits that can flow from explanation use, and how an explanation function should be constructed. Work with newer types of intelligent systems and help functions for everyday systems, such as word-processors, appears in many cases to neglect lessons learned in the past. This paper attempts to rectify this situation by drawing together the considerable body of work on the nature and use of explanations. Empirical studies, mainly with knowledge-based systems, are reviewed and linked to a sound theoretical base. The theoretical base combines a cognitive effort perspective, cognitive learning theory, and Toulmin's model of argumentation. Conclusions drawn from the review have both practical and theoretical significance. Explanations are important to users in a number of circumstances— when the user perceives an anomaly, when they want to learn, or when they need a specific piece of knowledge to participate properly in problem solving. Explanations, when suitably designed, have been shown to improve performance and learning and result in more positive user perceptions of a system. The design is important, however, because it appears that explanations will not be used if the user has to exert "too much" effort to get them. Explanations should be provided automatically if this can be done relatively unobtrusively, or by hypertext links, and should be context-specific rather than generic. Explanations that conform to Toulmin's model of argumentation, in that they provide adequate justification for the knowledge offered, should be more persuasive and lead to greater trust, agreement, satisfaction, and acceptance—of the explanation and possibly also of the system as a whole.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“While knowledge-based systems can provide clear explanations of the reasoning behind their outputs (Wick & Slagle, 1989; Gregor & Benbasat, 1999), models trained by machine learning algorithms may not (Lipton, 2015).”
From Algorithmic Accountability and Public Reason (2018) · cites this paper as Gregor & Benbasat, 1999
In the citing paper
“Receivers may require explanations about how the AI software came to a decision, especially when they must take action based on that decision (Gregor and Benbasat 1999; Watson et al. 2019).”
From How to Design AI for Social Good: Seven Essential Factors (2020) · cites this paper as Gregor and Benbasat 1999
In the citing paper
“Empirical studies (Gregor and Benbasat 1999) suggest that the suitability of an explanation differs among receivers according to their expertise.”
From How to Design AI for Social Good: Seven Essential Factors (2020) · cites this paper as Gregor and Benbasat 1999
Citations
Cites
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
User Acceptance of Computer Technology: A Comparison of Two Theoretical Models
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989) · Management Science · doi:10.1287/mnsc.35.8.982 · Cited by 25,709
A BEHAVIORAL MODEL OF RATIONAL CHOICE
Simon, H. A. (1955) · The Quarterly Journal of Economics · doi:10.2307/1884852 · Cited by 15,877
Moore, G. C., & Benbasat, I. (1991) · Information Systems Research · doi:10.1287/isre.2.3.192 · Cited by 9,150
Self-citationHuman agency in social cognitive theory.
Bandura, A. (1989) · American Psychologist · doi:10.1037/0003-066x.44.9.1175 · Cited by 8,263
Toulmin, S. E. (2003) · doi:10.1017/cbo9780511840005 · Cited by 7,101
Axten, N., Newell, A., & Simon, H. A. (1973) · Contemporary Sociology · doi:10.2307/2063712 · Cited by 6,834
Anderson, J. R. (2013) · doi:10.4324/9781315799438 · Cited by 6,761
Newell, A. (1972) · Cited by 6,222
Rational choice and the structure of the environment.
Simon, H. A. (1956) · Psychological Review · doi:10.1037/h0042769 · Cited by 5,919
Strategies of Discourse Comprehension
Staczek, J. J., van Dijk, T. A., & Kintsch, W. (1985) · The Modern Language Journal · doi:10.2307/326517 · Cited by 5,395
Categorization and Representation of Physics Problems by Experts and Novices*
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981) · Cognitive Science · doi:10.1207/s15516709cog0502_2 · Cited by 5,248
Johnson-Laird, P. N. (1983) · Cited by 5,190
Cognitive Psychology and Its Implications
Anderson, J. R. (2014) · Cited by 4,121
Acquisition of cognitive skill
Anderson, J. R. (1982) · Psychological Review · doi:10.1037/0033-295x.89.4.369 · Cited by 3,663
Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993) · doi:10.1017/cbo9781139173933 · Cited by 3,487
Leave, C. (1994) · Journal of the Operational Research Society · doi:10.1057/jors.1994.133 · Cited by 3,309
Rempel, J. K., Holmes, J. G., & Zanna, M. P. (1985) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.49.1.95 · Cited by 3,146
Perception and the Representative Design of Psychological Experiments
Brunswik, E. (1956) · doi:10.1525/9780520350519 · Cited by 2,974
Computer-Based Medical Consultations: MYCIN.
(1976) · Annals of Internal Medicine · doi:10.7326/0003-4819-85-6-831_1 · Cited by 2,502
Agents that reduce work and information overload
Maes, P. (1994) · Communications of the ACM · doi:10.1145/176789.176792 · Cited by 2,321
Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology
Chao, Y. R., & Zipf, G. K. (1950) · Language · doi:10.2307/409735 · Cited by 2,193
Galbraith, J. R. (1977) · Cited by 2,180
Craik, K. J. W. (1943) · Cited by 1,845
Psychology of Reasoning: Structure and Content
Wason, P. C., & Johnson-Laird, P. N. (1972) · Medical Entomology and Zoology · Cited by 1,784
Interpersonal trust, trustworthiness, and gullibility.
Rotter, J. B. (1980) · American Psychologist · doi:10.1037/0003-066x.35.1.1 · Cited by 1,694
Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology.
Hollingshead, A. B., & Zipf, G. K. (1949) · American Sociological Review · doi:10.2307/2086695 · Cited by 1,355
Dedre Gentner, C. (1983) · Cited by 1,329
Cognitive Conceptions of Learning
Shuell, T. J. (1986) · Review of Educational Research · doi:10.2307/1170340 · Cited by 1,116
Genesereth, M. R., & Ketchpel, S. P. (1994) · Communications of the ACM · doi:10.1145/176789.176794 · Cited by 1,092
Ross, L., Lepper, M. R., & Hubbard, M. (1975) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.32.5.880 · Cited by 1,052
The cognitive psychology of school learning
Gagné, E. D., Yekovich, C. W., & Yekovich, F. R. (1985) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 926
Toulmin, S. E., Rieke, R. D., & Janik, A. S. (1979) · Cited by 923
Clancey, W. J. (1985) · Artificial Intelligence · doi:10.1016/0004-3702(85)90016-5 · Cited by 880
Johnson, E. J., & Payne, J. W. (1985) · Management Science · doi:10.1287/mnsc.31.4.395 · Cited by 829
The epistemology of a rule-based expert system —a framework for explanation
Clancey, W. J. (1983) · Artificial Intelligence · doi:10.1016/0004-3702(83)90008-5 · Cited by 712
Cognitive psychology and its implications, 2nd ed.
Anderson, J. R. (1985) · Cited by 708
FITTS, P. M. (1964) · Categories of Human Learning · doi:10.1016/b978-1-4832-3145-7.50016-9 · Cited by 657
Effects of cognitive feedback on performance.
Balzer, W. K., Doherty, M. E., & O'Connor, R. (1989) · Psychological Bulletin · doi:10.1037//0033-2909.106.3.410 · Cited by 516
Decision support and expert systems
Turban, E. (1993) · Cited by 478
Constructive Interaction and the Iterative Process of Understanding
Miyake, N. (1986) · Cognitive Science · doi:10.1207/s15516709cog1002_2 · Cited by 445
XPLAIN: a system for creating and explaining expert consulting programs
Swartout, W. R. (1983) · Artificial Intelligence · doi:10.1016/s0004-3702(83)80014-9 · Cited by 393
Problem solving and the development of abstract categories in programming languages
Adelson, B. (1981) · Memory & Cognition · doi:10.3758/bf03197568 · Cited by 385
(1987) · Cited by 378
Expert judgment: Some necessary conditions and an example.
Einhorn, H. J. (1974) · Journal of Applied Psychology · doi:10.1037/h0037164 · Cited by 342
Computational Models of Discourse
(1983) · The MIT Press eBooks · doi:10.7551/mitpress/2019.001.0001 · Cited by 341
Paradigm Shifts in Designed Instruction: From Behaviorism to Cognitivism to Constructivism.
Cooper, P. A. (1993) · Educational Technology archive · Cited by 307
The Impact of Explanation Facilities on User Acceptance of Expert Systems Advice
Ye, L. R., & Johnson, P. E. (1995) · MIS Quarterly · doi:10.2307/249686 · Cited by 290
Expert systems: the next challenge for managers
Luconi, F. L., Malone, T. W., & Morton, M. S. S. (1986) · Sloan management review · doi:10.5555/8991.8992 · Cited by 286
Knowledge-based tutoring: the GUIDON program
Clancey, W. J. (1987) · CERN Document Server (European Organization for Nuclear Research) · Cited by 265
Todd, P., & Benbasat, I. (1991) · Information Systems Research · doi:10.1287/isre.2.2.87 · Cited by 253
Self-citationKnowledge management systems: emerging views and practices from the field
Alavi, M., & Leider, D. (2003) · Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences. 1999. HICSS-32. Abstracts and CD-ROM of Full Papers · doi:10.1109/hicss.1999.772754 · Cited by 250
Towards an understanding of the role of experience in the evolution from novice to expert
Kolodner, J. L. (1983) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(83)80068-6 · Cited by 229
What Kind Of Expert Should A System Be?
Johnson, P. E. (1983) · Journal of Medicine and Philosophy · doi:10.1093/jmp/8.1.77 · Cited by 218
Cognitive science and behaviourism
Skinner, B. F. (1985) · British Journal of Psychology · doi:10.1111/j.2044-8295.1985.tb01953.x · Cited by 217
Deep versus compiled knowledge approaches to diagnostic problem-solving
Chandrasekaran, B., & Mittal, S. (1983) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(83)80064-9 · Cited by 196
Dhaliwal, J. S., & Benbasat, I. (1996) · Information Systems Research · doi:10.1287/isre.7.3.342 · Cited by 194
Self-citationExpertise in auditing: Myth or reality?
Bédard, J. (1989) · Accounting, Organizations and Society · doi:10.1016/0361-3682(89)90037-8 · Cited by 194
Mayer, R. E. (1980) · Journal of Educational Psychology · doi:10.1037//0022-0663.72.6.770 · Cited by 184
Early expert systems: Where are they now?
Gill, T. G. (1995) · MIS Quarterly · doi:10.2307/249711 · Cited by 181
Interface design issues for advice-giving expert systems
Carroll, J. M., & McKendree, J. (1987) · Communications of the ACM · doi:10.1145/7885.7886 · Cited by 173
The state of knowledge-based systems
Hayes-Roth, F., & Jacobstein, N. (1994) · Communications of the ACM · doi:10.1145/175247.175249 · Cited by 172
Deep versus compiled knowledge approaches to diagnostic problem-solving
CHANDRASEKARAN, B., & MITTAL, S. (1999) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1983.0314 · Cited by 164
Explaining control strategies in problem solving
Chandrasekaran, B., Tanner, M. C., & Josephson, J. R. (1989) · IEEE Expert · doi:10.1109/64.21896 · Cited by 142
Survey of expert critiquing systems: practical and theoretical frontiers
Silverman, B. G. (1992) · Communications of the ACM · doi:10.1145/129852.129861 · Cited by 127
Intelligent Systems for Business: Expert Systems with Neural Networks
Zahedi, F. M. (1993) · Cited by 114
Huber, O. (1980) · Acta Psychologica · doi:10.1016/0001-6918(80)90031-1 · Cited by 109
Intelligent Interface Design: An Empirical Assessment of Knowledge Presentation in Expert Systems1
Lamberti, D. M., & Wallace, W. A. (1990) · MIS Quarterly · doi:10.2307/248891 · Cited by 99
Structural Analysis of Science Prose: Can We Increase Problem-Solving Performance?
Mayer, R. E. (2017) · Routledge eBooks · doi:10.4324/9781315099958-3 · Cited by 99
The Process of Question Answering -- A Computer Simulation of Cognition
Colclough, M., & Lehnert, W. G. (1979) · The Journal of the Operational Research Society · doi:10.2307/3009325 · Cited by 98
Evaluating user interfaces to information retrieval systems: a case study on user support
Brajnik, G., Mizzaro, S., & Tasso, C. (1996) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/243199.243249 · Cited by 98
Explanation and Verbalization in a Computer-Assisted Search Task
Berry, D. C., & Broadbent, D. E. (1987) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640748708401804 · Cited by 95
Expert Systems: A Decision Support Approach.
Edwards, J. S., Klein, M., & Methlie, L. B. (1991) · Journal of the Operational Research Society · doi:10.1057/jors.1991.86 · Cited by 81
Communicative acts for explanation generation
Maybury, M. T. (1992) · International Journal of Man-Machine Studies · doi:10.1016/0020-7373(92)90083-w · Cited by 81
Surprisingness and expectation failure: what's the difference?
Ortony, A., & Partridge, D. (1987) · International Joint Conference on Artificial Intelligence · Cited by 75
Explanation facilities and interactive systems
Johnson, H., & Johnson, P. (1993) · Proceedings of the 1st international conference on Intelligent user interfaces - IUI '93 · doi:10.1145/169891.169951 · Cited by 71
An explanation facility for today's expert systems
Wick, M. R., & Slagle, J. R. (1989) · IEEE Expert · doi:10.1109/64.21897 · Cited by 68
On making expert systems more like experts
SWARTOUT, W. R., & SMOLIAR, S. W. (1987) · Expert Systems · doi:10.1111/j.1468-0394.1987.tb00143.x · Cited by 68
Selection criteria for expert system shells: a socio-technical framework
Stylianou, A. C., Madey, G. R., & Smith, R. D. (1992) · Communications of the ACM · doi:10.1145/135239.135240 · Cited by 61
ExperTAXsm: an expert system for corporate tax planing
Shpilberg, D. C., Graham, L., & SCHATZ, H. (1986) · Expert Systems · doi:10.1111/j.1468-0394.1986.tb00487.x · Cited by 59
An intelligent assistant for patient health care
Miksch, S., Cheng, K., & Hayes‐Roth, B. (1997) · doi:10.1145/267658.267770 · Cited by 56
Expert systems and the man–machine interface
Berry, D. C., & Broadbent, D. E. (1986) · Expert Systems · doi:10.1111/j.1468-0394.1986.tb00204.x · Cited by 55
Expert systems and the man‐machine interface. Part Two: The user interface.
BERRY, D. C., & BROADBENT, D. E. (1987) · Expert Systems · doi:10.1111/j.1468-0394.1987.tb00408.x · Cited by 53
A Learning Curve Analysis of Expert System Use*
Fedorowicz, J., Oz, E., & Berger, P. D. (1992) · Decision Sciences · doi:10.1111/j.1540-5915.1992.tb00420.x · Cited by 43
Human cognition and the expert system interface: mental models and inference explanations
Rook, F. W., & Donnell, M. L. (1993) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/21.257760 · Cited by 42
Decision support and expert systems (4th ed.): management support systems
Turban, E. (1995) · Prentice-Hall, Inc eBooks · Cited by 41
Expert critics in engineering design: lessons learned and research needs
Silverman, B. G., & Mezher, T. M. (1992) · AI Magazine · doi:10.1609/aimag.v13i1.977 · Cited by 40
Artificial Intelligence: What Works and What Doesn't?
Hayes-Roth, F. A. (1997) · AI Magazine · doi:10.1609/aimag.v18i2.1296 · Cited by 38
What do users ask? Some thoughts on diagnostic advice
Kidd, A. L. (1986) · Cited by 37
The value of explanation in expert systems for auditing: An experimental investigation
Ye, L. R. (1995) · Expert Systems with Applications · doi:10.1016/0957-4174(95)00023-2 · Cited by 35
An experimental investigation of the use of explanations provided by knowledge-based systems
Dhaliwal, J. (2008) · cIRcle (University of British Columbia) · doi:10.14288/1.0086429 · Cited by 32
Combining discourse strategies to generate descriptions to users along a naive/expert spectrum
Paris, C. L. (1987) · International Joint Conference on Artificial Intelligence · Cited by 31
Reasoning and natural explanation
Goguen, J. A., Weiner, J. L., & Linde, C. (1983) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(83)80070-4 · Cited by 30
Gilbert, N. (1989) · The Knowledge Engineering Review · doi:10.1017/s026988890000504x · Cited by 26
Question classification in rule-based systems
Shaw, M. L. G., & Gaines, B. R. (1987) · Cited by 25
The Turing effect: the nature of trust in expert systems advice
Lerch, F. J., Prietula, M. J., & Kulik, C. T. (1997) · The MIT Press eBooks · Cited by 23
Contextualized access to knowledge: theoretical perspectives and a process‐tracing study
Mao, J., & Benbasat, I. (1998) · Information Systems Journal · doi:10.1046/j.1365-2575.1998.00034.x · Cited by 17
Self-citationAn Analysis of the Pedagogical Effects of Expert System Use in the Classroom
Moffitt, K. (1994) · Decision Sciences · doi:10.1111/j.1540-5915.1994.tb00813.x · Cited by 17
An explanatory and “argumentative” interface for a model-based diagnostic system
Miller, C. A., & Larson, R. (1992) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/142621.142627 · Cited by 17
On the Nature of Explanation: A PDP Approach
Churchland, P. M. (1995) · Boston studies in the philosophy of science · doi:10.1007/978-94-011-0461-6_6 · Cited by 16
Cognitive support: designing aiding to supplement human knowledge
de Greef, H. P., & Neerincx, M. A. (1995) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1995.1023 · Cited by 15
Decision support and expert systems : management support systems, 4th ed
Turban, E. (1995) · Cited by 15
Michener, E. R. (1984) · European Conference on Artificial Intelligence · Cited by 12
The partitioned support network for expert system justification
Wick, M. R., & Slagle, J. R. (1989) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/21.31059 · Cited by 11
On the nature of explanation: A PDP approach
Churchland, P. M. (1990) · Physica D Nonlinear Phenomena · doi:10.1016/0167-2789(90)90083-2 · Cited by 11
Moffitt, K., & Hershauer, J. C. (1989) · Cited by 11
Enhanced maintenance and explanation of expert systems through explicit models of their development
Neches, R. T., Swartout, W., & Moore, J. D. (1984) · Cited by 11
An Analysis of the Pedagogical Effects of Expert System Use in the Classroom
Moffitt, K. (1994) · Decision Sciences · doi:10.1111/j.1540-5915.1994.tb01852.x · Cited by 11
Enhancing explanations in knowledge‐based systems with hypertext
Mao, J., Benbasat, I., & Dhaliwal, J. (1996) · Journal of Organizational Computing and Electronic Commerce · doi:10.1080/10919399609540279 · Cited by 10
Self-citationEverett, A. M. (1994) · Lincoln (University of Nebraska) · Cited by 9
Hsu, K.-C. (1993) · University Microfilms International eBooks · Cited by 9
Liebowitz, J. (1997) · AI Magazine · doi:10.1609/aimag.v18i2.1297 · Cited by 9
Mao, J. (2009) · Open Collections · doi:10.14288/1.0088358 · Cited by 7
The explanation facility and the explanation effect
Swinney, L. S. (1995) · Expert Systems with Applications · doi:10.1016/0957-4174(95)00024-0 · Cited by 7
User requirements for explanation in expert systems
Johnson, P. E., Davis, G. B., & Ye, L. R. (1990) · Cited by 7
Improving the Use of Knowledge-Based Systems with Explanations
Brézillon, P., & Cawsey, A. J. (1992) · AI Communications · doi:10.3233/aic-1992-5409 · Cited by 4
Eining, M. M. (1987) · SHAREOK (University of Oklahoma; Oklahoma State University; Central Oklahoma University) · Cited by 2
Dynamic Function Allocation in Fighter Cockpits
Aretz, A. J., Hickox, J. C., & Kesler, S. R. (1987) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193128703100407 · Cited by 2
A personal financial planning system for everyday use?
Gregor, S. D. (1996) · Springer eBooks · doi:10.1007/978-0-387-34979-4_22 · Cited by 1
Self-citationLearning and explanation type in a knowledge-based arms control assistant: an empirical evaluation
Gault, R. W. (1994) · Cited by 1
Research works printed but not linked: 26
- Anderson, 1985, Cognitive Psychology and Its Implications
- Ausubel, 1985, Learning as Constructing Meaning, 71
- Best, 1989, Cognitive Psychology
- Bruner, 1956, A Study of Thinking
- Brunswik, 1952, The Conceptual Foundation of Psychology
- Buchanan, 1984, Expert Systems
- Eining, 1991, The Impact of Expert System Usage on Experiential Learning in an Auditing Setting, Journal of Information Systems, 5, 1
- 1995, Expert Systems with Applications, 8
- Fitts, 1976, Human Performance
- Flowler, 1985, The Relationship Between Cognitive Style and Dialogue Style
- Gregor, S. D. Explanations from Knowledgebased Systems for Human Learning and Problem Solving, unpublished doctoral dissertation, University of Queensland, Brisbane, Australia, 1996a.
- Gregor, 1997, Explanations from Knowledgebased Systems and the Learning versus Working Argument
- Gregor, 1997, Knowledge-based Systems in Prescriptive versus Supportive Roles and the Use of Explanations
- Kidd, 1985, The Consultative Role of an Expert System, 228
- Koonce, 1992, Explanation and Counterexplanation During Audit Analytical Review, The Accounting Review, 67, 59
- Lerch, 1993, Measuring Trust in Machine Advice
- 1981, The Macquarie Dictionary
- Mao, 1996, The Effects of Hypertext-based Explanations in Knowledgebased Systems on Explanation Use and Knowledge Transfer
- Mao, 1996, Exploring the Use of Explanations in Knowledge-based Systems: A Process Tracing Analysis
- Murphy, 1990, Expert System Use and the Development of Expertise in Auditing: A Preliminary Investigation, Journal of Information Systems, 18
- Oltman, 1971, Group Embedded Figures Test
- Schank, 1986, Explanation: A First Pass, 139
- Schroder, 1967, Human Information Processing
- Seligman, 1990, Learned Optimism
- Wognum, P. M. Explanation of Automated Reasoning: How and Why?unpublished doctoral dissertation, University of Twente, Enschede, The Netherlands, 1990.
- Yourdon, 1989, Modern Structure Analysis
Cited by
The Nature of Theory in Information Systems1
Gregor, S. (2006) · MIS Quarterly · doi:10.2307/25148742 · Cited by 3,306
“…(see, for example, Achinstein 1983; Craik 1943; Hempel and Oppenheim 1948; Nagel 1979), in relation to everyday reasoning and the nature of argumentation (see Toulmin 1958; Toulmin et al. 1979), as well as in the subbranches of IS relating to knowledge-based systems (see Gregor and Benbasat 1999).”
BackgroundSelf-citationSynthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987
E-Commerce Product Recommendation Agents: Use, Characteristics, and Impact1
Xiao, B., & Benbasat, I. (2007) · MIS Quarterly · doi:10.2307/25148784 · Cited by 1,081
“…to involve actual use of an RA (prototype or operational, web-based or stand-alone) by human users (similar to the definition of empirical studies in Gregor and Benbasat 1999) in either online or physical settings.8 • The study must have dependent variables that go beyond “accuracy” and…”
BackgroundInfluentialSelf-citationQuestioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
“n kind of explanation...to satisfy certain user purposes of user goals. ” In the early generation of AI work, question-driven frameworks were used to generate explanations for knowledge-based systems [20,38,88]. AQUA [72] is a reasoning model that uses questions to generate explanations and identify knowledge gaps for…”
MethodManaging artificial intelligence
Berente, N., Gu, B., Recker, J., & Santhanam, R. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16274 · Cited by 898
Metadata
- Authors
- Gregor, S., & Benbasat, I.
- Affiliations
- Central Queensland University (AU)
- University of British Columbia (CA)
- Venue
- MIS Quarterly
- Publisher
- MIS Quarterly
- Published
- 1999-12-01
- Volume
- 23
- Issue
- 4
- Pages
- 497-530
- ISSN
- 0276-7783 (ISSN-L)
- 2162-9730
- OpenAlex
- W1576620340
- Cited by
- 535
- References
- 122
- Citations by year
- 2026: 28 · 2025: 31 · 2024: 20 · 2023: 47 · 2022: 22 · 2021: 43 · 2020: 32 · 2019: 30 · 2018: 21 · 2017: 21 · 2016: 13 · 2015: 11 · 2014: 17 · 2013: 6 · 2012: 22
- FWCI
- 7.04
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- MIS Quarterly (publishedVersion)
- Journal of the Association for Information Systems (submittedVersion)
- Funders
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- explanation generation, intelligent systems, expert systems, decision support system, intelligent agents, knowledge management systems, Toulmin model, cognitive effort, cognitive learning theory, explanation design, user trust, user satisfaction, system acceptance, word processing software
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.