The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Decomposing Complexity: Modeling NLP Systems for Crisis Sensemaking with Systemigrams
Abstract
Generative artificial intelligence (GenAI) tools, such as Large Language Models (LLMs), are increasingly deployed in high-stakes domains to support decision-making, situational awareness, and knowledge synthesis. Yet their integration into crisis workflows raises significant challenges related to interpretability, epistemic trust, and organizational alignment. Drawing on semi-structured interviews with intelligence and military professionals, this study investigates how practitioners evaluate and operationalize LLMs in environments defined by uncertainty and time pressure. Thematic analysis revealed three core tensions: Uncertainty and Mistrust of Black Box Systems in Crisis Situations, Interpretability and Operational Pressure, and Using AI to Support Sensemaking. These themes serve to motivate a systems-oriented approach to effectively integrating AI systems into sensemaking applications. Building on complexity theory and sensemaking literature, we propose a systemigram-based framework for modeling the interdependent components of NLP systems, Human, Prompt, LLM, Uncertainty, and Sensemaking, as a modular sociotechnical system. This framework provides a tool for both analyzing and designing NLP applications in crisis contexts, helping practitioners trace feedback loops, understand dependencies, and promote resilient, interpretable AI integration. Our findings underscore the importance of epistemic alignment and system-level transparency for effective human-AI collaboration in critical decision environments.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
The Architecture of Complexity
Simon, H. A. (1991) · Facets of Systems Science · doi:10.1007/978-1-4899-0718-9_31 · Cited by 4,433
ENACTED SENSEMAKING IN CRISIS SITUATIONS[1]
Weick, K. E. (1988) · Journal of Management Studies · doi:10.1111/j.1467-6486.1988.tb00039.x · Cited by 2,015
Better Zero-Shot Reasoning with Role-Play Prompting
Kong, A., Zhao, S., Chen, H., Li, Q., Qin, Y., Sun, R., Zhou, X., Wang, E., & Dong, X. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.228 · Cited by 171
Tian, K., Mitchell, E., Zhou, A., Sharma, A., Rafailov, R., Yao, H., Finn, C., & Manning, C. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.330 · Cited by 123
FROM SENSEMAKING IN ORGANIZATIONS
Weick, K. E. (2021) · Princeton University Press eBooks · doi:10.2307/j.ctv1f886rp.24 · Cited by 83
(2015) · SAGE Publications Ltd eBooks · doi:10.4135/9781412986281.n339 · Cited by 77
Comes, T. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09750-0 · Cited by 54
Goecks, V. G., & Waytowich, N. R. (2024) · International Conference on Military Communications and Information Systems (ICMCIS) · doi:10.1109/icmcis61231.2024.10540749 · Cited by 45
Research works printed but not linked: 11
- Ancona, 2012, Framing and acting in the unknown, The handbook for teaching leadership: Knowing, doing, and being, 3
- Balasubramani, 2024, Crafting narratives: Real-time generative storytelling through tangible AI. PhD thesis
- Burns, 2023, Weak-to-strong generalization: Eliciting strong capabilities with weak supervision
- Herm, 2023, Impact of explainable ai on cognitive load: Insights from an empirical study, arXiv preprint
- Le, 2019, Menestrel and Kenneth W Kizer, Strengthening the military family readiness system for a changing american society
- Liu, 2024, Uncertainty estimation and quantification for llms: A simple supervised approach, arXiv preprint
- Schafer, 2015, Characteristics of individuals and employment among first responders, US Department of Labor
- Weaver, 1948, Science and complexity, American scientist, 36, 536
- Yao, 2023, React: Synergizing reasoning and acting in language models
- Yue, 2023, Large language model cascades with mixture of thoughts representations for cost-efficient reasoning, arXiv preprint
- Zheng, 2023, Secrets of rlhf in large language models part i: Ppo, arXiv preprint
Cited by
Still being checked.
Metadata
- Authors
- Farr, D., Clark, N., Ng, L. H. X., Zhu, J. H., Gueorguieva, A.-M., West, J., & Starbird, K.
- Affiliations
- University of Washington (US)
- Carnegie Mellon University (US)
- University of Maryland, College Park (US)
- ORCID
- Venue
- 2025 IEEE International Symposium on Systems Engineering (ISSE)
- Publisher
- IEEE
- Published
- 2025-10-28
- Pages
- 1-8
- OpenAlex
- W7127952605
- Semantic Scholar
- 4e66c80f0a2fea64f4ba703fa17e7c87c1f2817e
- Cited by
- 0
- References
- 12
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Symposium on Systems Engineering (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Military Strategy and Technology
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Decision Sciences › Management Science and Operations Research › Complex Systems and Decision Making
- Keywords
- large language models, systemigram, socio-technical systems, interpretability, epistemic trust, human-AI collaboration, uncertainty, military decision-making, intelligence analysis, complexity theory, feedback loops
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Decomposing Complexity: Modeling NLP Systems for Crisis Sensemaking with Systemigrams. https://app.cochat.ai/papers/10.1109/isse65546.2025.11370061
- The paper
- Farr, D., Clark, N., Ng, L. H. X., Zhu, J. H., Gueorguieva, A.-M., West, J., & Starbird, K. (2025). Decomposing Complexity: Modeling NLP Systems for Crisis Sensemaking with Systemigrams. 2025 IEEE International Symposium on Systems Engineering (ISSE). https://doi.org/10.1109/isse65546.2025.11370061
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
