Get To The Point! Problem-Based Curated Data Views To Augment Care For Critically Ill Patients
Open accessAbstract
Electronic health records in critical care medicine offer unprecedented opportunities for clinical reasoning and decision making. Paradoxically, these data-rich environments have also resulted in clinical decision support systems (CDSSs) that fit poorly into clinical contexts, and increase health workers cognitive load. In this paper, we introduce a novel approach to designing CDSSs that are embedded in clinical workflows, by presenting problem-based curated data views tailored for problem-driven discovery, team communication, and situational awareness. We describe the design and evaluation of one such CDSS, In-Sight, that embodies our approach and addresses the clinical problem of monitoring critically ill pediatric patients. Our work is the result of a co-design process, further informed by empirical data collected through formal usability testing, focus groups, and a simulation study with domain experts. We discuss the potential and limitations of our approach, and share lessons learned in our iterative co-design process.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Transparency and explainability regarding AI actions are crucial for enabling humans to appropriately trust and engage with AI contributions.
“Transparency and explainability regarding AI actions [70, 187, 209] are crucial for enabling humans to appropriately trust and engage with AI contributions [191, 201].”
From Exploring Collaboration Patterns and Strategies in Human-AI Co-creation through the Lens of Agency: A Scoping Review of the Top-tier HCI Literature (2025) · cites this paper as [201]
The citation is attached to the claim about transparency and explainability of AI actions. Section 7.2 discusses limiting computerized annotations to preserve clinicians’ reasoning agency, and the conclusion describes machine-learning assistance as future work; neither addresses transparency or explanations of AI actions. Searches for “transparency” and “explainability” found no passage supporting that point. Page numbers were unavailable in our copy. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Institute of Medicine (2000) · National Academies Press eBooks · doi:10.17226/9728 · Cited by 14,174 · see citing works
An overview of clinical decision support systems: Benefits, risks, and strategies for success
Sutton, R. T., Pincock, D., Baumgart, D. C., Sadowski, D. C., Fedorak, R. N., & Kroeker, K. I. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0221-y · Cited by 3,019 · see citing works
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936 · see citing works
The “Meaningful Use” Regulation for Electronic Health Records
Blumenthal, D., & Tavenner, M. (2010) · New England Journal of Medicine · doi:10.1056/nejmp1006114 · Cited by 2,059 · see citing works
Bates, D. W., Kuperman, G. J., Wang, S., Gandhi, T., Kittler, A., Volk, L., Spurr, C., Khorasani, R., Tanasijevic, M., & Middleton, B. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1370 · Cited by 1,346 · see citing works
Han, Y. Y., Carcillo, J. A., Venkataraman, S. T., Clark, R. S. B., Watson, R. S., Nguyen, T. C., Bayir, H., & Orr, R. A. (2005) · Pediatrics · doi:10.1542/peds.2005-1287 · Cited by 1,149 · see citing works
A roadmap for national action on clinical decision support
Osheroff, J. A., Teich, J. M., Middleton, B., Steen, E. B., Wright, A., & Detmer, D. E. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2334 · Cited by 707 · see citing works
LifeLines: visualizing personal histories
Catherine Plaisant, B., Milash, B. A., Rose, A., Widoff, S., & Shneiderman, B. (1996) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/238386.238493 · Cited by 597 · see citing works
Grand challenges in clinical decision support
Sittig, D. F., Wright, A., Osheroff, J. A., Middleton, B., Teich, J. M., Ash, J. S., Campbell, E., & Bates, D. W. (2008) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.09.003 · Cited by 582 · see citing works
Clinical Decision-Support Systems
Musen, M. A., Blackford Middleton, A., & Greenes, R. A. (2013) · Springer eBooks · doi:10.1007/978-1-4471-4474-8_22 · Cited by 577 · see citing works
Dynamic queries for visual information seeking
Shneiderman, B. (1994) · IEEE Software · doi:10.1109/52.329404 · Cited by 574 · see citing works
Messmer, A. S., Zingg, C., Müller, M., Gerber, J. L., Schefold, J. C., & Pfortmueller, C. A. (2020) · Critical Care Medicine · doi:10.1097/ccm.0000000000004617 · Cited by 468 · see citing works
Laxmisan, A., Hakimzada, F., Sayan, O. R., Green, R. A., Zhang, J., & Patel, V. L. (2007) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2006.09.019 · Cited by 449 · see citing works
Computer Technology and Clinical Work: Still Waiting for Godot
Wears, R. L., & Berg, M. (2005) · JAMA · doi:10.1001/jama.293.10.1261 · Cited by 448 · see citing works
Development of medical checklists for improved quality of patient care
Hales, B., Terblanche, M., Fowler, R., & Sibbald, W. J. (2007) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzm062 · Cited by 375 · see citing works
Recommender systems in the healthcare domain: state-of-the-art and research issues
Tran, T. N. T., Felfernig, A., Trattner, C., & Holzinger, A. (2020) · Journal of Intelligent Information Systems · doi:10.1007/s10844-020-00633-6 · Cited by 305 · see citing works
Improving Outcomes with Clinical Decision Support : An Implementer's Guide
Osheroff, J. A., Teich, J. M., Levick, D., Saldana, L., Velasco, F. T., Sittig, D. F., Rogers, K. M., & Jenders, R. A. (2012) · Cited by 297 · see citing works
Yang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247 · see citing works
Phansalkar, S., Edworthy, J., Hellier, E., Seger, D. L., Schedlbauer, A., Avery, A. J., & Bates, D. W. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.005264 · Cited by 226 · see citing works
Alarm fatigue: impacts on patient safety
Ruskin, K. J., & Hueske-Kraus, D. (2015) · Current Opinion in Anaesthesiology · doi:10.1097/aco.0000000000000260 · Cited by 226 · see citing works
Aligning temporal data by sentinel events
Wang, T. D., Catherine Plaisant, B., Quinn, A. J., Stanchak, R., Murphy, S. N., & Shneiderman, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357129 · Cited by 220 · see citing works
Graphical summary of patient status
Powsner, S. M. (1994) · The Lancet · doi:10.1016/s0140-6736(94)91406-0 · Cited by 216 · see citing works
The Art and Practice of Systems Biology in Medicine: Mapping Patterns of Relationships
van der Greef, J., Martin, S. A., Juhász, P., Adourian, A. S., Plasterer, T. N., Verheij, E. R., & McBurney, R. N. (2007) · Journal of Proteome Research · doi:10.1021/pr0606530 · Cited by 200 · see citing works
Interactive Information Visualization to Explore and Query Electronic Health Records
Rind, A., Wang, T. D., Wolfgang Aigner, A., Miksch, S., Wongsuphasawat, K., Catherine Plaisant, B., & Shneiderman, B. (2013) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000039 · Cited by 199 · see citing works
The effects of safety checklists in medicine: a systematic review
Thomassen, Ø., Storesund, A., Søfteland, E., & Brattebø, G. (2013) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.12207 · Cited by 184 · see citing works
Jacobs, M., He, J., F. Pradier, M., Lam, B., Ahn, A. C., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445385 · Cited by 177 · see citing works
Bellomo, R., Ronco, C., Mehta, R. L., Asfar, P., Helms, J., Darmon, M., Diehl, J., Duranteau, J., Hoste, E. A. J., Olivier, J.-B., et al. (2017) · Annals of Intensive Care · doi:10.1186/s13613-017-0260-y · Cited by 171 · see citing works
Connecting time-oriented data and information to a coherent interactive visualization
Bade, R., Schlechtweg, S., & Miksch, S. (2004) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985706 · Cited by 148 · see citing works
Dashboard visualizations: Supporting real-time throughput decision-making
Franklin, A., Gantela, S., Shifarraw, S., Johnson, T. R., Robinson, D. J., King, B. R., Mehta, A. M., Maddow, C. L., Hoot, N. R., Nguyen, V., et al. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.05.024 · Cited by 126 · see citing works
TimeLine: Visualizing Integrated Patient Records
Bui, A. A. T., Aberle, D. R., & Kangarloo, H. (2007) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2006.884365 · Cited by 125 · see citing works
The Craft of Information Visualization
Bederson, B. B., & Shneiderman, B. 1. (2003) · Elsevier eBooks · doi:10.1016/b978-1-55860-915-0.x5000-8 · Cited by 112 · see citing works
A Systematic Review of Natural Language Processing in Healthcare
Iroju, O. G., & Janet, O. (2015) · International Journal of Information Technology and Computer Science · doi:10.5815/ijitcs.2015.08.07 · Cited by 111 · see citing works
Electronic Medical Records and Physician Productivity: Evidence from Panel Data Analysis
Bhargava, H. K., & Mishra, A. N. (2014) · Management Science · doi:10.1287/mnsc.2014.1934 · Cited by 101 · see citing works
Medical Question Answering for Clinical Decision Support
Goodwin, T. R., & Harabagiu, S. M. (2016) · doi:10.1145/2983323.2983819 · Cited by 70 · see citing works
Fluid Creep and Over-resuscitation
Saffle, J. R. (2016) · Critical Care Clinics · doi:10.1016/j.ccc.2016.06.007 · Cited by 69 · see citing works
Reasoning processes in clinical reasoning: from the perspective of cognitive psychology
Shin, H. S. (2019) · Korean journal of medical education · doi:10.3946/kjme.2019.140 · Cited by 60 · see citing works
More Text Please! Understanding and Supporting the Use of Visualization for Clinical Text Overview
Sultanum, N., Brudno, M., Wigdor, D., & Chevalier, F. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173996 · Cited by 57 · see citing works
Self-citationClinician Cognitive Overload and Its Implications for Nurse Leaders
Collins, R. (2020) · Nurse Leader · doi:10.1016/j.mnl.2019.11.007 · Cited by 57 · see citing works
Malnutrition in the ICU Patient Population
Powers, J., & Samaan, K. H. (2014) · Critical Care Nursing Clinics of North America · doi:10.1016/j.ccell.2014.01.003 · Cited by 52 · see citing works
Winters-Miner, L. A., Bolding, P. S., Hilbe, J. M., Goldstein, M., Hill, T., Nisbet, R., Walton, N., & Miner, G. D. (2014) · Elsevier eBooks · doi:10.1016/b978-0-12-411643-6.00003-x · Cited by 41 · see citing works
What's the Purpose of Rounds? A Qualitative Study Examining the Perceptions of Faculty and Students
Hulland, O., Farnan, J. M., Rabinowitz, R., Kearns, L., Long, M., Monash, B., Bhansali, P., & Fromme, H. B. (2017) · Journal of Hospital Medicine · doi:10.12788/jhm.2835 · Cited by 36 · see citing works
Checklists: A Review of Their Origins, Benefits, and Current Uses as a Cognitive Aid in Medicine
Chaparro, A., Keebler, J. R., Lazzara, E. H., & Diamond, A. (2019) · Ergonomics in Design The Quarterly of Human Factors Applications · doi:10.1177/1064804618819181 · Cited by 36 · see citing works
Baron, J. M., Huang, R., McEvoy, D. S., & Dighe, A. S. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab006 · Cited by 36 · see citing works
Assessing EHR use during hospital morning rounds: A multi-faceted study
Assis-Hassid, S., Grosz, B. J., Zimlichman, E., Rozenblum, R., & Bates, D. W. (2019) · PLoS ONE · doi:10.1371/journal.pone.0212816 · Cited by 29 · see citing works
Use and impact of a computer-generated patient summary worksheet for primary care.
Wilcox, A. B., Jones, S. S., Dorr, D. A., Cannon, W., Burns, L., Radican, K., Christensen, K., Brunker, C. P., Larsen, A., Narus, S. P., et al. (2005) · PubMed Central · Cited by 26 · see citing works
Patient Development at a Glance: An Evaluation of a Medical Data Visualization
Pohl, M., Wiltner, S., Rind, A., et al. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23768-3_24 · Cited by 26 · see citing works
PhenoBlocks: Phenotype Comparison Visualizations
Glueck, M., Hamilton, P., Chevalier, F., et al. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2467733 · Cited by 26 · see citing works
Self-citationBiomedical Data: Their Acquisition, Storage, and Use
Shortliffe, E. H., & Barnett, G. O. (2006) · Computers in health care · doi:10.1007/0-387-36278-9_2 · Cited by 23 · see citing works
Modeling and Leveraging Analytic Focus During Exploratory Visual Analysis
Zhou, Z., Wen, X., Wang, Y., et al. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445674 · Cited by 21 · see citing works
The Review of Systems, the Electronic Health Record, and Billing
Hendrickson, M. A., Melton, G. B., & Pitt, M. B. (2019) · JAMA · doi:10.1001/jama.2019.5667 · Cited by 20 · see citing works
Situational Awareness in Critical Care: An Aviation Approach to Reduce Error
Rodriguez, A. C., Lee, D. A., & Makic, M. B. F. (2017) · Journal of PeriAnesthesia Nursing · doi:10.1016/j.jopan.2017.08.001 · Cited by 7 · see citing works
Flagged first, then by citations
Research works printed but not linked: 9
- [9]9. Andreas Buja, John Alan McDonald, John Michalak, and Werner Stuetzle. 1991. Interactive data visualization using focusing and linking. In Proceedings of the 2nd conference on Visualization'91. 156-163. https://doi.org/10.1109/VISUAL. 1991.175794 · DOI
- [10]10. Mackinlay Card. 1999. Readings in information visualization: using vision to think. Morgan Kaufmann. https://doi.org/doi/10.5555/300679 · DOI
- [12]12. James J Cimino, Anthony Aguirre, Stephen B Johnson, and Ping Peng. 1993. Generic queries for meeting clinical information needs. Bulletin of the Medical Library Association 81, 2 (1993), 195. https://www.ncbi.nlm.nih.gov/pmc/articles/ PMC225762/
- [16]16. Anthony Faiola, Preethi Srinivas, and Jon Duke. 2015. Supporting clinical cog- nition: a human-centered approach to a novel ICU information visualization dashboard. In AMIA Annual Symposium Proceedings, Vol. 2015. American Medical Informatics Association, 560. https://pubmed.ncbi.nlm.nih.gov/26958190/
- [24]24. Joy Higgs and Mark A Jones. 2008. Clinical decision making and multiple problem spaces. Clinical reasoning in the health professions 3 (2008), 3-17.
- [44]44. Sandeep Sharma, Muhammad F Hashmi, and Rotem Friede. 2020. Interprofes- sional rounds in the ICU. StatPearls [Internet] (2020). https://pubmed.ncbi.nlm. nih.gov/29939553/
- [50]50. Nicole Sultanum, Devin Singh, Michael Brudno, and Fanny Chevalier. 2018. Doccurate: A Curation-Based Approach for Clinical Text Visualization. IEEE transactions on visualization and computer graphics 25, 1 (2018), 142-151. http: //www.cs.toronto.edu/~fanny/pubs/pdf/doccurate-vast2018.pdf
- [54]54. Stacey L Valentine, Anil Sapru, Renee A Higgerson, Phillip C Spinella, Heidi R Flori, Dionne A Graham, Molly Brett, Maureen Convery, LeeAnn M Christie, Laurie Karamessinis, et al. 2012. Fluid balance in critically ill children with acute lung injury. Critical care medicine 40, 10 (2012), 2883.
- [56]56. Qiru Wang and RS Laramee. 2021. EHR STAR: The State-Of-the-Art in Interactive EHR Visualization. In Computer Graphics Forum. Wiley Online Library.
Cited by
Sorted by citations
Zhang, S., Wang, H., & Yi, X. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757594 · Cited by 36 · see citing works
Mastrianni, A., Kim, M. S., Sullivan, T. M., Sippel, G. J., Burd, R. S., Gajos, K. Z., & Sarcevic, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757512 · Cited by 6 · see citing works
- Authors
- Zhang, M., Ehrmann, D., Mazwi, M., Eytan, D., Ghassemi, M., & Chevalier, F.
- Affiliations
- University of Toronto (CA)
- Hospital for Sick Children (CA)
- Massachusetts Institute of Technology (US)
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-13
- OpenAlex
- W4225093325
- Cited by
- 16
- References
- 51
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 2 · 2023: 7
- FWCI
- 3.25
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
- Keywords
- clinical decision support systems, intensive care, pediatric intensive care, electronic health records, clinical workflow integration, situational awareness, team communication, cognitive load, co-design, usability evaluation, simulation study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
