The DOI matches its Crossref record; 112 of the 115 references it cites were found in a registry or an index; none of the 45 sources its claims cite is checked for support yet.
Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory
Open accessAbstract
Understanding how ML models work is a prerequisite for responsibly designing, deploying, and using ML-based systems. With interpretability approaches, ML can now offer explanations for its outputs to aid human understanding. Though these approaches rely on guidelines for how humans explain things to each other, they ultimately solve for improving the artifact—an explanation. In this paper, we propose an alternate framework for interpretability grounded in Weick’s sensemaking theory, which focuses on who the explanation is intended for. Recent work has advocated for the importance of understanding stakeholders’ needs—we build on this by providing concrete properties (e.g., identity, social context, environmental cues, etc.) that shape human understanding. We use an application of sensemaking in organizations as a template for discussing design guidelines for sensible AI, AI that factors in the nuances of human cognition when trying to explain itself.
Overview
CoChat Summary
TL;DR
The paper argues that current interpretability and explainability approaches focus on improving explanations as artifacts and therefore fall short of helping people reliably use ML-based systems. It proposes grounding interpretability in Weick’s sensemaking theory and introduces Sensible AI, a framework using seven sensemaking properties and high-reliability organization principles as design guidelines.
Key findings
- Current interpretability and explainability solutions “will always fall short” because they focus on designing better explanations rather than understanding how humans make sense of complex, sometimes conflicting information; sensemaking subsumes interpretability and includes contextual factors such as identity, social and organizational context, and prior experience.
- The paper translates Weick’s seven sensemaking properties—identity construction, social, retrospective, enactive, ongoing, focused on extracted cues, and plausibility over accuracy—into claims for the human-machine context, e.g., people internalize explanations that support their identity, social context affects explanation effectiveness, providing explanations before reflection negatively affects sensemaking, explanation order affects understanding, interruptions and emotion change what is understood, highlighting different parts of explanations leads to varying understanding, and plausible explanations reduce the search for accurate ones.
- It proposes Sensible AI design guidelines inspired by high-reliability organizations: seamful design, inducing skepticism, adversarial design, and continuous monitoring and feedback, drawing on HRO principles such as preoccupation with failure, reluctance to simplify, sensitivity to operations, commitment to resilience, and deference to expertise.
- No original empirical numbers are reported; cited reviews include 289 core papers and 12,412 citing papers, and 409 papers taxonomized, but these are prior work, not the paper’s own findings.
Methods
- Conceptual/theoretical paper with no empirical study, dataset, or sample; it reviews interpretability and explainability literature and translates Weick’s sensemaking theory from organizational studies to the human-machine context.
- Uses an application of sensemaking in organizations and high-reliability organizations as a template, presenting example vignettes for each sensemaking property (e.g., healthcare diagnosis, content moderation, diabetic retinopathy, predictive policing, hiring, radiology).
- Analysis is argumentative and framework-building: it defines the seven properties, derives claims for human-machine sensemaking, and proposes design guidelines for Sensible AI.
Limitations
- The paper states the example vignettes are crafted based on popular press articles and research papers, are not intended as representative, and do not claim a causal relationship between the property and the example; other reasons for the described system behavior are possible.
- The paper presents “initial design ideas” and a “research agenda”; no empirical validation, user study, deployment, or evaluation of Sensible AI is reported (evident from the text).
- The paper notes there is not yet consensus on how interpretability and explainability differ from an implementation point of view, and it uses “interpretability” for the rest of the paper.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
![Fig. 1 Left: DARPA's conceptualization of Explainable AI, adapted from [35]. Right: Weick's sensemaking properties (1-7) categorized using the high-level Enactment-Selection-Retention organizational model, adapted from [49]. Enactment includes properties about perceiving and acting on environmental changes; Selection, properties related to interpreting what the changes mean; and Retention, properties that describe storing and using prior experiences [56]. Our Sensible AI framework extends the existing definition of interpretability and explainability to include Weick's sensemaking properties.](/paper-figures/0d084958fd6844c697ff6f4fc4d43b5c/sensible-ai-re-imagining-interpretability-and-explainability-using-sensemaking-figure-1.png)
![Fig. 2 Saliency maps for chest radiographs, adapted from [20].](/paper-figures/9b14734865984f4ca91dd08ba4d8c9e3/sensible-ai-re-imagining-interpretability-and-explainability-using-sensemaking-figure-2.png)
Verification
Claims the paper makes
0 Valid · 3 Not valid · 30 Not checked · 26 Own results and interpretations
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If the police officers were also provided an explanation for the model's predictions, the type of explanation and the order in which they are seen (e.g., global vs. local explanation first) changes the enacted environment for the officers.”
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > 3.4.2 Enactment in the Human-Machine Context.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
In the paper
“For example, accurate mental models and deliberative reasoning can help avoid ML practitioners' misuse of, and over-reliance on, interpretability outputs [52].”
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > 2.2 Understanding the "who" in interpretability and explainability
What the sources say
- Not matchedcites [52]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“Explanation interfaces that are interactive and collaborative can improve overall accuracy [108].”
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > 2.2 Understanding the "who" in interpretability and explainability
What the sources say
- Not matchedcites [108]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Because sensemaking is sensitive to enacted environments, it is important to remember that any information or explanation about the model is not treated by people as static or isolated.”
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > 3.4.2 Enactment in the Human-Machine Context.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“Weick notes the work on socially shared cognition (e.g., [62,91]) which shows that human cognition and social functioning are essential to each other.”
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > 3.2.1 Social Elements of Sensemaking in the
What the sources say
- Not matchedcites [62]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [91]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
22 more claims
Create a free account to see all 59 claims and every source verdict.
- Probably not a substantial claim. 20
- It may state two separate claims as one. 16
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > Abstract
“Weick's sensemaking is more procedural: "placement of items into frameworks, comprehending, redressing surprise, constructing meaning, interacting in pursuit of mutual understanding, and patterning" [118, p.6].”
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > References
“118. K.E. Weick, K.M. Sutcliffe, and D. Obstfeld. 1999. Organizing for high reliability: Processes of Collective Mindfulness. Research in Organizational Behaviour 21 (1999), 81-123.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > 3.2.1 Social Elements of Sensemaking in the
“recall Weick's reference to Mead's work on the individual being "a parliament of selves" [74]”
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > References
“74. George Mandler. 1984. Mind and body: Psychology of emotion and stress. WW Norton & Company Incorporated.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > 3.3.1 Retrospective Sensemaking in the Human-Human Context.
“"as I begin the latter portion of a long word, my utterance of the first part is already in the past" [36, p.44].”
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > References
“36. C. Hartshorne. 1962. Mind as Memory and Creative Love. In Theories of the Mind, Jordan M. Scher (Ed.). Free Press of Glencoe, 440-463.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > 3.5.1 Sensemaking as an Ongoing Activity in the
“Winograd and Flores describe these situations as having the following properties: (1) you cannot avoid acting; (2) you cannot step back and reflect on your actions, i.e., you have to rely on your intuitions; (3) the effects of action cannot be predicted;”
§ Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory > References
“123. Mark Wenman. 2013. Agonistic democracy: Constituent power in the era of globalisation. Cambridge University Press.”
Citations
Cites
Qualitative Data Analysis: An Expanded Sourcebook
(1994) · Journal of Environmental Psychology · doi:10.1016/s0272-4944(05)80231-2 · Cited by 48,420
James, W. (1913) · Henry Holt and Company eBooks · doi:10.1037/11059-000 · Cited by 25,226
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Grice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221
Quinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Gioia, D. A., Mehra, A., & Weick, K. E. (1996) · The Academy of Management Review · doi:10.2307/259169 · Cited by 15,614
Generalized Additive Models: An Introduction with R
Wood, S. N. (2017) · doi:10.1201/9781315370279 · Cited by 13,852
Telling more than we can know: Verbal reports on mental processes.
Nisbett, R. E., & Wilson, T. D. (1977) · Psychological Review · doi:10.1037/0033-295x.84.3.231 · Cited by 11,317
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Research works printed but not linked: 16
- Gordon W Allport. 1985. The historical background of social psychology (Vol. 1). The handbook of social psychology(1985).
- 10.18653/v1 · doi:10.18653/v1
- Ellen Berscheid. 1983. Emotion. In Close Relationships, H.H. Kelley, E. Berscheid, A. Christensen, J. Harvey, T. Huston, G. Levinger, E. McClintock, A. Peplau, and D.R. Peterson (Eds.). WH Freeman, 110–168.
- Wilhelm Dilthey and Frederic Jameson. 1972. The rise of hermeneutics. New literary history 3, 2 (1972), 229–244.
- C. Hartshorne. 1962. Mind as Memory and Creative Love. In Theories of the Mind, Jordan M. Scher (Ed.). Free Press of Glencoe, 440–463.
- Will Douglas Heaven. 2020. Predictive policing algorithms are racist. They need to be dismantled. MIT Technology Review 17(2020), 2020.
- Daniel J Isenberg. 1986. The structure and process of understanding: Implications for managerial action. In The Thinking Organization, H.P. Sims Jr. and D.A. Gioia (Eds.). Jossey Bass, San Fransisco, 238–262.
- P Devereaux Jennings and Royston Greenwood. 2003. Constructing the iron cage: Institutional theory and enactment. Debating organization: point-counterpoint in organization studies 195 (2003).
- Jeremy Kahn. 2021. HireVue drops facial monitoring amid A.I. algorithm audit. Fortune (2021).
- George Herbert Mead. 1934. Mind, self and society. Vol. 111. Chicago University of Chicago Press.
- Charles Sanders Peirce. 1878. Illustrations of the Logic of Science: IV The Probability of Induction. Popular Science Monthly 12 (Apr 1878), 705–718.
- Peter Pirolli and Stuart Card. 2005. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, Vol. 5. McLean, VA, USA, 2–4.
- Joseph C Pitt. 1988. Theories of explanation. Oxford University Press.
- Louis R Pondy and Ian I Mitroff. 1979. Beyond open system models of organization. Research in organizational behavior 1, 1 (1979), 3–39.
- Ramprasaath R Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization. In ICCV.
- K.E. Weick, K.M. Sutcliffe, and D. Obstfeld. 1999. Organizing for high reliability: Processes of Collective Mindfulness. Research in Organizational Behaviour 21 (1999), 81–123.
Cited by
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice
Delgado, F., Yang, S., Madaio, M., & Yang, Q. (2023) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3617694.3623261 · Cited by 256
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
Miller, T. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594001 · Cited by 172
Liao, Q. V., Subramonyam, H., Wang, J., & Vaughan, J. W. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580652 · Cited by 121
Harnessing Biomedical Literature to Calibrate Clinicians’ Trust in AI Decision Support Systems
Yang, Q., Hao, Y., Quan, K., Yang, S., Zhao, Y., Kuleshov, V., & Wang, F. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581393 · Cited by 91
Metadata
- Authors
- Kaur, H., Adar, E., Gilbert, E., & Lampe, C.
- Affiliations
- University of Michigan (US)
- University of Michigan, USA
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 702-714
- OpenAlex
- W4280636581
- Semantic Scholar
- 2ae9a9ec0c6a614d5bc522e64983cded8e6f4fb9
- Cited by
- 92
- References
- 115
- Citations by year
- 2026: 27 · 2025: 23 · 2024: 24 · 2023: 16 · 2022: 1
- FWCI
- 6.71
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Google (PhD Fellowship)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- explainable AI, sensemaking theory, Weick, stakeholder needs, social environment, environmental cues, human cognition, responsible AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-14, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-14 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
