CoChat
Sign Up
Cited by 34Electronic Health Records SystemsMachine Learning in HealthcareHealthcare Systems and Technology

Identifying collaborative care teams through electronic medical record utilization patterns

Open access
Chen, You, Lorenzi, Nancy M, Sandberg, Warren S, Wolgast, Kelly, Malin, Bradley A·2017·Journal of the American Medical Informatics Association
DOI 10.1093/jamia/ocw124PMC PMC6080725
Open this paper in CoChat — free accountOpen PDF

Abstract

Objective: The goal of this investigation was to determine whether automated approaches can learn patient-oriented care teams via utilization of an electronic medical record (EMR) system. Materials and Methods: To perform this investigation, we designed a data-mining framework that relies on a combination of latent topic modeling and network analysis to infer patterns of collaborative teams. We applied the framework to the EMR utilization records of over 10 000 employees and 17 000 inpatients at a large academic medical center during a 4-month window in 2010. Next, we conducted an extrinsic evaluation of the patterns to determine the plausibility of the inferred care teams via surveys with knowledgeable experts. Finally, we conducted an intrinsic evaluation to contextualize each team in terms of collaboration strength (via a cluster coefficient) and clinical credibility (via associations between teams and patient comorbidities). Results: The framework discovered 34 collaborative care teams, 27 (79.4%) of which were confirmed as administratively plausible. Of those, 26 teams depicted strong collaborations, with a cluster coefficient > 0.5. There were 119 diagnostic conditions associated with 34 care teams. Additionally, to provide clarity on how the survey respondents arrived at their determinations, we worked with several oncologists to develop an illustrative example of how a certain team functions in cancer care. Discussion: Inferred collaborative teams are plausible; translating such patterns into optimized collaborative care will require administrative review and integration with management practices. Conclusions: EMR utilization records can be mined for collaborative care patterns in large complex medical centers.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/jamia/ocw124

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Studies using raw audit logs fit into three broad categories: manual mapping, identification of networks, application of automatic methods.

    “Studies using raw audit logs fit into three broad categories:”

    From Electronic health record activity changes around new decision support implementation: monitoring using audit logs and topic modeling (2025) · cites this paper as [81]

    The Research Design and Methods and Conclusions sections describe an automated method for identifying collaborative networks from EMR utilization logs, so this paper is an example of one category. They do not classify audit-log studies into the three stated categories or discuss manual mapping as a category. The passages provided have no page numbers. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 36

  1. Peterson MW. Emerging developments in postsecondary organization the- ory and research: fragmentation or integration. Educ Res. 1985;14(3):5-12.
  2. Stange KC. The problem of fragmentation and the need for integrative so- lutions. Ann Fam Med. 2009;7(2):100-103.
  3. Richardson WC, Briere R, eds. Crossing the quality chasm: a new health system for the 21st century. Committee on Quality of Health Care in Amer- ica, Institute of Medicine. Washington, DC: National Academy Press; 2001.
  4. Fisher ES. Building a medical neighborhood for the medical home. N Engl J Med. 2008;359:1202-1205.
  5. Asarnow JR, Rozenman M, Wiblin J, Zeltzer L. Integrated medical- behavioral care compared with usual primary care for child and adolescent behavioral health: a meta-analysis. JAMA Pediatr. 2015;169(10):929-937.
  6. Cebul RD, Rebitzer JB, Taylor LJ, Votruba ME. Organizational fragmenta- tion and care quality in the U.S. healthcare system. J Eco Pers. 2008; 22:93-113.
  7. Reid RJ, Coleman K, Johnson EA, et al. The Group Health Medical Home at year two: cost savings, higher patient satisfaction, and less burnout for providers. Health Aff. 2010;29:835-843.
  8. Valderas JM, Starfield B, Sibbald B, Salisbury C, Roland M. Defining co- morbidity: implications for understanding health and health services. Ann Fam Med. 2009;7:357-363.
  9. Rittenhouse DR, Shortell SM. The patient-centered medical home: will it stand the test of health reform? JAMA. 2009;301(19):2038-2040.
  10. Bergeson SC, Dean JD. A systems approach to patient-centered care. JAMA. 2006;296(23):2848-2851.
  11. Grumbach K, Bodenheimer T. Can health care teams improve primary care practice? JAMA. 2004;291:1246-1251.
  12. Wagner EH. The role of patient care teams in chronic disease manage- ment. BMJ. 2000;320:569-572.
  13. Berwick D. What patient-centered should mean: confessions of an extrem- ist. Health Aff. 2009;28(4):w555-w565.
  14. Chen Y, Nyemba S, Malin B. Auditing medical records accesses via health- care interaction networks. AMIA Annu Symp. 2012;2012:93-102.
  15. Carley K, Lee J. Dynamic organizations: organizational adaptation in a changing environment. Adv Stra Manage. 1998;15:269-297.
  16. Carley K. Computational organization science: a new frontier. PNAS. 2002;99(3):7257-7262
  17. Merrill J, Bakken S, Rockoff M, Gebbie K, Carley K. Description of a method to support public health information management: organizational network analysis. J Biomed Inform. 2007;40:422-428.
  18. Cunningham FC, Ranmuthugala G, Plumb J, Georgiou A, Westbrook J, Braithwaite J. Health professional networks as a vector for improving healthcare quality and safety: a systematic review. BMJ Qual Saf. 2012;21:239-249.
  19. Uddin S, Hossain L, Hamra J, Alam A. A study of physician collaborations through social network and exponential random graph. BMC Health Serv Res. 2013;13(234):1-14.
  20. Merill JA, Sheehan BM, Carley KM, Stetson PD. Transition networks in a co- hort of patients with congestive heart failure. Appl Clin Inform. 2015;6:548-564.
  21. Chen Y, Ghosh J, Bejan CA, et al. Building bridges across electronic health record systems through inferred phenotypic topics. J Biomed Inform. 2015;55:82-93.
  22. Friedman CP, Wong AK, Blumenthal D. Achieving a nationwide learning health system. Sci Transl Med. 2010;2(57):57cm29.
  23. Kho AN, Pacheco JA, Peissig PL, et al. Electronic Medical Records for Genetic Research: Results of the eMERGE Consortium. Sci Transl Med. 2011;3(79):79re1.
  24. Wu S, Chaudhry B, Wang J, et al. Systematic review: impact of health in- formation technology on quality, efficiency, and costs of medical care. Ann Int Med. 2006;144(10):742-752.
  25. Unertl KM, Johnson KB, Lorenzi NM. Health information exchange tech- nology on the front lines of healthcare: workflow factors and patterns of use. J Am Med Inform Assoc. 2012;19:392-400.
  26. Chen Y, Lorenzi N, Nyemba S, Schildcrout JS, Malin B. We work with them? Health workers' interpretation of organizational relations mined from electronic health records. Int J Med Inform. 2014;83:495-506.
  27. Peissig P, Costa VS, Caldwell MD, et al. Relational machine learning for electronic health record driven phenotyping. J Biomed Inform. 2014;52:260-270.
  28. Tanpowpong P, Broder-Fingert S, Obuch JC, Rahni DO, Katz AJ, Leffler DA. Multicenter study on the value of ICD-9-CM codes for case identifi- cation of celiac disease. Ann Epidemiol. 2013;203:136-142.
  29. Denny JC, Ritchie MD, Basford MA, et al. PheWAS: demonstrating the feasibility of a phenome-wide scan to discover gene-disease associations. Bioinformatics. 2010;26:1205-1210.
  30. Denny JC, Bastarache L, Ritchie MD, et al. Systematic comparison of phenome-wide association study of electronic medical record data and genome-wide association study data. Nat Biotechnol. 2013;31:1102-1111.
  31. Chen Y, Nyemba S, Malin B. Detecting anomalous insiders in collaborative information systems. IEEE Trans Dependable Secure Comput. 2012;9: 332-344.
  32. Pivovarov R, Perotte AJ, Grave E, Angiolillo J, Wiggins CH, Elhadad N. Learning probabilistic phenotypes from heterogeneous EHR data. J Biomed Inform. 2015;58:156-165
  33. Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. J Mac Learn Res. 2003;3:993-1022.
  34. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics sup- port. J Biomed Inform. 2009;42:377-381.
  35. David C, Hoaglinab E. The hat matrix in regression and ANOVA. The Am Stat. 1978;32(1):17-2240.
  36. Watts DJ, Strogatz S. Collective dynamics of small-world networks. Nature. 1998;393:440-442.

Cited by

Sorted by citations

Authors
Chen, Y., Lorenzi, N. M., Sandberg, W. S., Wolgast, K., & Malin, B. A.
Affiliations
Vanderbilt University (US)
Corresponding author
Chen, You
ORCID
Venue
Journal of the American Medical Informatics Association
Publisher
Oxford University Press (OUP)
Published
2016-08-28
Volume
24
Issue
e1
Pages
e111-e120
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
DOI
10.1093/jamia/ocw124
PMC
PMC6080725
OpenAlex
W2516936793
Cited by
34
References
2
Citations by year
2026: 1 · 2025: 3 · 2024: 1 · 2023: 1 · 2022: 3 · 2021: 6 · 2020: 2 · 2019: 6 · 2018: 5 · 2017: 4 · 2016: 2
FWCI
9.65
Type
journal article
Language
en
Open access
Locations
  • Journal of the American Medical Informatics Association (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Funders
National Institutes of Health (R00LM011933 and R01LM010685)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Healthcare Systems and Technology
Keywords
electronic health records, topic modeling, network analysis, data mining, clustering coefficient, clinical credibility, comorbidities, cancer care, academic medical centers
MeSH terms
Cooperative Behavior, Humans, Interprofessional Relations, Patient Care Team, Patient-Centered Care, Data Mining, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review