Cited by 265Explainable Artificial Intelligence (XAI)Text and Document Classification TechnologiesTopic Modeling

The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.

Explaining Data-Driven Document Classifications1

Martens, David, Provost, Foster·2014·MIS Quarterly

DOI 10.25300/misq/2014/38.1.04

Open this paper in CoChat — free account

Abstract

Many document classification applications require human understanding of the reasons for data-driven classification decisions by managers, client-facing employees, and the technical team. Predictive models treat documents as data to be classified, and document data are characterized by very high dimensionality, often with tens of thousands to millions of variables (words). Unfortunately, due to the high dimensionality, understanding the decisions made by document classifiers is very difficult. This paper begins by extending the most relevant prior theoretical model of explanations for intelligent systems to account for some missing elements. The main theoretical contribution is the definition of a new sort of explanation as a minimal set of words (terms, generally), such that removing all words within this set from the document changes the predicted class from the class of interest. We present an algorithm to find such explanations, as well as a framework to assess such an algorithm’s performance. We demonstrate the value of the new approach with a case study from a real-world document classification task: classifying web pages as containing objectionable content, with the goal of allowing advertisers to choose not to have their ads appear on those pages. A second empirical demonstration on news-story topic classification shows the explanations to be concise and document-specific, and to be capable of providing understanding of the exact reasons for the classification decisions, of the workings of the classification models, and of the business application itself. We also illustrate how explaining the classifications of documents can help to improve data quality and model performance.

Integrity signals

Registry

Crossref · 10.25300/misq/2014/38.1.04

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Untitled work
(1992) · Cited by 23,431
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537
Fast algorithms for mining association rules
Agrawal, R., & Srikant, R. (1998) · Cited by 11,362
Design Science in Information Systems Research1
Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004) · MIS Quarterly · doi:10.2307/25148625 · Cited by 10,707 · cited 1×
“We now present an empirical case study (Hevner et al. 2004) onthe problem of classifying web pages as containing adult content.”
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,123
Fast Algorithms for Mining Association Rules in Large Databases
Agrawal, R., & Srikant, R. (1994) · Very Large Data Bases · Cited by 9,454
Text categorization with Support Vector Machines: Learning with many relevant features
Joachims, T. (1998) · Lecture notes in computer science · doi:10.1007/bfb0026683 · Cited by 8,271

Show all 68 references

Research works printed but not linked: 2

Buchanan, 1984, Rule Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project
Hettich, S., and Bay, S. D. 1996. “The UCI KDD Archive” (https://kdd.ics.uci.edu).

Sorted by citations

Items 201–265 of 265

Design Principles for Explainable Sales Win-Propensity Prediction Systems
Thiess, T., Müller, O., Tonelli, L., & MAN Energy Solutions, A. (2020) · doi:10.30844/wi_2020_c8-thiess · Cited by 3
Model-Agnostic Explanations using Minimal Forcing Subsets
Han, X., & Ghosh, J. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533992 · Cited by 3
The Application of Gradient Boost Machine and Local Interpretable Modelagnostic Explanations in Examining the Travel Multi-mode Choice, the Case Study of Jakarta City, Indonesia
Ha, T. V., ASADA, T., & Arimura, M. (2019) · Journal of the Eastern Asia Society for transportation studies/Journal of the Eastern Asia Society for Transportation Studies · doi:10.11175/easts.13.503 · Cited by 3
We Need to Understand Where the Data Comes From: Co-Designing Transparent AI Systems for Caregiving
Hermann, J., Jansen, N., & Doğangün, A. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708557.3716354 · Cited by 2
Supervised feature compression based on counterfactual analysis
Piccialli, V., Morales, D. R., & Salvatore, C. (2023) · European Journal of Operational Research · doi:10.1016/j.ejor.2023.11.019 · Cited by 4
A General Search-Based Framework for Generating Textual Counterfactual Explanations
Gilo, D., & Markovitch, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29764 · Cited by 4
Contextualized LORE for Fuzzy Attributes
Maaroof, N., Moreno, A., Jabreel, M., & Valls, A. (2021) · Frontiers in artificial intelligence and applications · doi:10.3233/faia210164 · Cited by 2
Learning to Learn to Predict Performance Regressions in Production at Meta
Beller, M., Li, H., Nair, V., Murali, V., Ahmad, I., Cito, J., Carlson, D., Aye, G. A., & Dyer, W. (2023) · doi:10.1109/ast58925.2023.00010 · Cited by 9
On Fairness and Calibration
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603198 · Cited by 2
Algorithmic Monoculture and Social Welfare
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603211 · Cited by 2
Contextual Local Explanation for Black Box ClassifiersPreprint
Zhang, Z., Yang, F., Wang, H., & Hu, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00768 · Cited by 2
A Series of Unfortunate Counterfactual Events: the Role of Time in Counterfactual ExplanationsPreprint
Ferrario, A., & Loi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.04687 · Cited by 2
Towards Explaining STEM Document Classification using Mathematical Entity LinkingPreprint
Scharpf, P., Schubotz, M., & Gipp, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.00954 · Cited by 2
IFME: Influence Function Based Model Explanation for Black Box Decision Systems
Zha, B., & Shen, H. (2020) · Communications in computer and information science · doi:10.1007/978-981-15-2767-8_27 · Cited by 1
Causal Visual Feature Extraction for Image Classification Interpretation
Bao, C., Chen, D., Wang, M., & Pan, Q. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191952 · Cited by 1
Improving Customer Churn Prediction: A Study of Counterfactual Explanations Using Wachter's Method, Growing Spheres Method, and Genetic Algorithms
Singh, A. K., & Tripathi, A. K. (2025) · International Conference on Communication, Security and Artificial Intelligence (ICCSAI) · doi:10.1109/iccsai64074.2025.11064483 · Cited by 1
Enhancing counterfactual explanations with causal inference: integrating DirectLiNGAM and variational autoencoders for constraint-aware generation
Dinh, D., Nguyen, T.-A., Nguyen, N.-T., Le, N., & Le, B. H. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06945-w · Cited by 1
The impact of the automated ball-strike system (ABS) on baseball: A deep learning based counterfactual approach
Hwang, S., Kim, J., & Lee, S. (2025) · Proceedings of the Institution of Mechanical Engineers Part P Journal of Sports Engineering and Technology · doi:10.1177/17543371251395162 · Cited by 1
Concept-Based Text Mining Technique for Semantic Classification of Manufacturing Suppliers
Shotorbani, P. Y., & Ameri, F. (2017) · Smart and Sustainable Manufacturing Systems · doi:10.1520/ssms20160005 · Cited by 1
Explanations as Discourse
Afrashteh, S., Someh, I., & Davern, M. J. (2020) · AJIS. Australasian journal of information systems/AJIS. Australian journal of information systems/Australian journal of information systems · doi:10.3127/ajis.v24i0.2519 · Cited by 1
NED
Gujral, E., Neves, L., Papalexakis, E. E., & Shah, N. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482455 · Cited by 1
[Construction of a multi-copy Pichia expression vector and expression of human brain-derived neurotrophic factor in Pichia pastoris].
Liang, W., Zhang, C.-C., & Xicai, Y. (2005) · PubMed · Cited by 1
Automated Image Reduction for Explaining Black-box Classifiers
Jiang, M., Tang, C., Zhang, X., Zhao, Y., & Ding, Z. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00042 · Cited by 1
ComplAI: Framework for Multi-factor Assessment of Black-Box Supervised Machine Learning Models
De, A., Gudipudi, S. S., Panchanan, S., & Desarkar, M. S. (2023) · ACM Symposium on Applied Computing · doi:10.1145/3555776.3577771 · Cited by 2
Lymphoma Ultrasound Image Classification with Causal Attention and Feature Fusion
Chen, D., Han, Y., Dong, Y., & Zhou, X. (2022) · IEEE Smart World Congress (SWC) · doi:10.1109/smartworld-uic-atc-scalcom-digitaltwin-pricomp-metaverse56740.2022.00068 · Cited by 1
Mitigating Bias in Algorithmic Decision-making: Evaluating Claims and Practices
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603212 · Cited by 1
Assessing the Economic Benefits of E- Commerce Adoption for Small and Medium Enterprises (SMEs)
(2024) · International Journal of Progressive Research in Engineering Management and Science · doi:10.58257/ijprems35760 · Cited by 2
Explaining LLM Decisions: Counterfactual Chain-of-Thought Approach
Sarkar, P., Prakash, A. V., & Singh, J. B. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-96473-2_18 · Cited by 1
Information Technology Infrastructure: A Source of Entrepreneurs’ Economic Challenges
Palmer, W. (2020) · Lecture notes in information systems and organisation · doi:10.1007/978-3-030-34269-2_7 · Cited by 1
The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603204 · Cited by 1
INFORMS Journal on Data Science (IJDS) Editorial #1: What Is an IJDS Paper?
Galit Shmueli, A. (2021) · INFORMS Journal on Data Science · doi:10.1287/ijds.2020.003 · Cited by 1
Intellige: A User-Facing Model Explainer for Narrative Explanations.Preprint
Yang, J., Negoescu, D. M., & Ahammad, P. (2021) · arXiv (Cornell University) · Cited by 1
Operational Risk Identification: An Effective and Explainable Design ApproachPreprint
Yang, Y., Zhang, K., & Zhou, F. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3603702 · Cited by 1
Local post-hoc interpretability for black-box classifiers
Laugel, T. (2020) · theses.fr (ABES) · doi:10.70675/afa1c80bz1e1fz4930zb0eaz215e7bb5bf65 · Cited by 1
Algorithmic monoculture and social welfare
(2023) · The Societal Impacts of Algorithmic Decision-Making · doi:10.1145/3603195.3603202
The Predictive Power of Massive Data about our Fine-Grained Behavior
Provost, F. J. (2016) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/2835776.2835846
Self-citation
Text Mining Techniques for Analyzing Unstructured Manufacturing Data
Shotorbani, P. Y. (2016)
On the Inherent Explainability of Pattern Theory-Based Video Event Interpretations
Aakur, S. N., de Souza, F. D. M., & Sarkar, S. (2018) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-98131-4_11
Comparison of classification model and annotation method for Undiksha’s official documents
Paramartha, A. A. G. Y., Marti, N. W., & Aryanto, K. Y. E. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1516/1/012026
Designing Causal Inference Systems for Value-Based Spare Parts Pricing
Thiess, T., & Müller, O. (2020) · Lecture notes in business information processing · doi:10.1007/978-3-030-61140-8_13
Sensing the Future: Designing Sensor-based Predictive Information Systems for Forecasting Spare Part Demand for Diesel Engines
Furtak, S. (2018) · Econstor (Econstor)
Absolute Value Constraint: The Reason for Invalid Performance Evaluation Results of Neural Network Models for Stock Price PredictionPreprint
Wei, Y. M. (2021) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2101.10942
Neural Combinatorial Optimization with Explanation
Liu, Z., & Duan, Q. (2022) · Neural Processing Letters · doi:10.1007/s11063-022-11028-9
Machine Learning Approach in the Indian Service Industry
Priyadarshinee, P. (2023) · doi:10.1002/9781119905172.ch10
Bibliography
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603213
Future Directions
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603205
Introduction
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603196
How Do Classifiers Induce Agents to Behave Strategically?
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603201
How Do Classifiers Induce Agents to Behave Strategically?
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603210
Authors’ Biographies/Index
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603214
What If the Prompt Were Different? Counterfactual Explanations for the Characteristics of Generative Outputs
Goethals, S., Sedoc, J., & Provost, F. J. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3708319.3733656 · Cited by 1
Self-citation
Creating Value from Enterprise AI
Someh, I., & Wixom, B. H. (2025) · Springer eBooks · doi:10.1007/978-3-032-01940-0_8
Inherent Tradeoffs in the Fair Determination of Risk Scores
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603206
The Externalities of Exploration and How Data Diversity Helps Exploitation
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603199
Selection Problems in the Presence of Implicit Bias
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603200
Human-Centered XAI in Elderly Care: A Communication-Focused Participatory Framework
Hermann, J., & Doğangün, A. (2025) · International Conference on Information Technology for Social Good (GoodIT) · doi:10.1145/3748699.3749814
Multi-Objective Optimization of Explainable AI Models Under Resource Constraints in Embedded Systems
Dhatterwal, J. S., & Dipesh (2025) · International Conference on Electronics and Computing, Communication Networking Automation Technologies (ICEC2NT) · doi:10.1109/icec2nt65402.2025.11380049
Deriving Business Value From Artificial Intelligence In Business-To-Consumer E-Commerce Firms
Bawack, R. E. (2021) · theses.fr (ABES) · doi:10.70675/8638d810zbddcz4e78z98f2zce3722f3e6f0
A Neural-Symbolic learning framework to produce interpretable predictions for image classification
Bennetot, A. (2022) · theses.fr (ABES) · doi:10.70675/2b59f598z2028z49afz8a20z29b21f19f394
Generality and explainability in recommender systems
Zhong, J. (2023) · theses.fr (ABES) · doi:10.70675/de5345e1z9ed3z4064zaaf4zc45a557c0632
Explainability for machine learning models : from data adaptability to user perception
Delaunay, J. (2023) · theses.fr (ABES) · doi:10.70675/933366c4z576ez464cz9fcfz24627f86dd2f
An Introduction to Explainable Artificial Intelligence Applications for Industrial-Organizational Psychology
Pitcher, B. D., & Wang, P. (2026) · Oxford University Press eBooks · doi:10.1093/9780197807309.003.0027
Fair Recourse for All: Ensuring Individual and Group Fairness in Counterfactual Explanations
Ezzeddine, F., Ammar, O., Giordano, S., & Ayoub, O. (2026) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2026.3688207 · Cited by 2
Exploiting explanations for model extraction via knowledge distillation and mitigation with private counterfactuals
Ezzeddine, F., Giordano, S., & Ayoub, O. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1746910 · Cited by 5
GraphExVis: visual analytics for GNN interpretability and trustworthiness via local explanations
Yu, C., Jiang, H., Tao, Y., Zhang, S., Yuan, Q., & Xu, J. (2026) · Journal of Visualization · doi:10.1007/s12650-026-01149-x

Authors: Martens, D., & Provost, F.
Affiliations: University of Antwerp (BE)
New York University (US)
ORCID: Martens, David
Provost, Foster
Venue: MIS Quarterly
Publisher: MIS Quarterly
Published: 2014-03-01
Volume: 38
Issue: 1
Pages: 73-100
ISSN: 0276-7783 (ISSN-L)
2162-9730
DOI: 10.25300/misq/2014/38.1.04
OpenAlex: W198744195
Cited by: 264
References: 68
Citations by year: 2026: 4 · 2025: 19 · 2024: 17 · 2023: 49 · 2022: 28 · 2021: 52 · 2020: 29 · 2019: 17 · 2018: 13 · 2017: 12 · 2016: 12 · 2015: 4 · 2014: 6
FWCI: 11.30
Type: journal article
Language: en
Open access: closed
Locations: MIS Quarterly (publishedVersion)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords: explainable AI, document classification, high-dimensional text data, web page classification, model interpretability, data quality improvement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Explaining Data-Driven Document Classifications1

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks