Barriers to Achieving Economies of Scale in Analysis of EHR Data: A Cautionary Tale
Open accessAbstract
Summary: Signed in 2009, the Health Information Technology for Economic and Clinical Health Act infused $28 billion of federal funds to accelerate adoption of electronic health records (EHRs). Yet, EHRs have produced mixed results and have even raised concern that the current technology ecosystem stifles innovation. We describe the development process and report initial outcomes of a chronic kidney disease analytics application that identifies high-risk patients for nephrology referral. The cost to validate and integrate the analytics application into clinical workflow was $217,138. Despite the success of the program, redundant development and validation efforts will require $38.8 million to scale the application across all multihospital systems in the nation. We address the shortcomings of current technology investments and distill insights from the technology industry. To yield a return on technology investments, we propose policy changes that address the underlying issues now being imposed on the system by an ineffective technology business model. Citation: Sendak MP, Balu S, Schulman KH. Barriers to Achieving Economies of Scale in Analysis of EHR Data. Appl Clin Inform 2017; 8: 826–831 https://doi.org/10.4338/ACI-2017-03-CR-0046
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Integrating a single AI model into a workflow costs over $200,000.
“With an estimated cost of over $200,000 to integrate a single AI model into a workflow 36 , such pre-deployment analyses are essential.”
From Standing on FURM Ground: A Framework for Evaluating Fair, Useful, and Reliable AI Models in Health Care Systems (2024) · cites this paper as [36]
The Abstract reports $217,138 to validate and integrate a CKD analytics application into clinical workflow. The Results clarify that this was the total pilot cost, including $130,000 for application design and development—not solely the cost of integrating a single AI model. The paper therefore supports the amount but not that characterization of what it paid for. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
George, H., Duke Jon, D., Shah Nigam, H., Reich Christian, G., Vojtech, H., Schuemie Martijn, J., Suchard Marc, A., Woong, P. R., Kei, W. I. C., Rijnbeek Peter, R., et al. (2015) · Studies in Health Technology and Informatics · doi:10.3233/978-1-61499-564-7-574 · Cited by 1,107 · see citing works
Hersh, W. R., Weiner, M. G., Embi, P. J., Logan, J. R., Payne, P. R. O., Bernstam, E. V., Lehmann, H. P., Hripcsak, G., Hartzog, T. H., Cimino, J. J., et al. (2013) · Medical Care · doi:10.1097/mlr.0b013e31829b1dbd · Cited by 567 · see citing works
Implications of an emerging EHR monoculture for hospitals and healthcare systems
Koppel, R., & Lehmann, C. U. (2015) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2014-003023 · Cited by 140 · cited 1× · see citing works
“Health care systems are locked into EHRs that lack data model standardization [4], and the business model persists whereby EHRs control all data rather than liberate data for use in innovative applications [5].”
Background
Flagged first, then by citations
Research works printed but not linked: 11
- Blumenthal D, Glaser JP. Information technology comes to medicine. N Engl J Med 2007; 356(24): 2527-2534.
- Bates DW, Saria S, Ohno-Machado L, Shah A, Escobar G. Big Data In health care: using analytics to ident- ify and manage high-risk and high-cost patients. Health Aff (Millwood) 2014; 33(7): 1123-1131.
- Mandl KD, Kohane IS. Escaping the EHR trap -the future of health IT. N Engl J Med 2012; 366(24): 2240-2242.
- Lee BJ, Forbes K. The role of specialists in managing the health of populations with chronic illness: the example of chronic kidney disease. BMJ 2009; 339: b2395.
- Lee B, Turley M, Meng D, Zhou Y, Garrido T, Lau A, Radler L. Effects of proactive population-based neph- rologist oversight on progression of chronic kidney disease: a retrospective control analysis. BMC Health Serv Res 2012; 12(1): 1.
- Fung V, Brand RJ, Newhouse JP, Hsu J. Using Medicare data for comparative effectiveness research: oppor- tunities and challenges. Am J Manag Care 2011; 17(7): 488-496.
- Adamusiak T, Shimoyama N, Shimoyama M. Next generation phenotyping using the unified medical lan- guage system. JMIR Med Inform 2014; 2(1): e5.
- Tangri N, Stevens LA, Griffith J, Tighiouart H, Djurdjev O, Naimark D, Levin A, Levey AS. A predictive model for progression of chronic kidney disease to kidney failure. JAMA 2011; 305(15): 1553-1559.
- Coresh J, Turin TC, Matsushita K, Sang Y, Ballew SH, Appel LJ, Arima H, Chadban SJ, Cirillo M, Djurdjev O, Green JA, Heine GH, Inker LA, Irie F, Ishani A, Ix JH, Kovesdy CP, Marks A, Ohkubo T, Shalev V, Shan- kar A, Wen CP, de Jong PE, Iseki K, Stengel B, Gansevoort RT, Levey AS; CKD Prognosis Consortium. De- cline in estimated glomerular filtration rate and subsequent risk of end-stage renal disease and mortality. JAMA 2014; 311(24): 2518-2531.
- Drawz PE, Archdeacon P, McDonald CJ, Powe NR, Smith KA, Norton J, Williams DE, Patel UD, Narva A. CKD as a model for improving chronic disease care through electronic health records. Clin J Am Soc Nephrol 2015; 10(8): 1-12.
- Redman TC. Improve data quality for competitive advantage. Sloan Manag Rev 1995; 36(2):99-107.
Cited by
Sorted by citations
Sendak, M. P., Ratliff, W., Sarro, D., Alderton, E., Futoma, J., Gao, M., Nichols, M., Revoir, M., Yashar, F., Miller, C., et al. (2020) · JMIR Medical Informatics · doi:10.2196/15182 · Cited by 287 · see citing works
Self-citationCorey, K. M., Kashyap, S., Lorenzi, E., Lagoo-Deenadayalan, S. A., Heller, K., Whalen, K., Balu, S., Heflin, M. T., McDonald, S. R., Swaminathan, M., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002701 · Cited by 204 · see citing works
Self-citationA Path for Translation of Machine Learning Products into Healthcare Delivery
(2020) · EMJ Innovations · doi:10.33590/emjinnov/19-00172 · Cited by 172 · see citing works
Self-citationGuo, L. L., Pfohl, S. R., Fries, J., Johnson, A. E. W., Posada, J., Aftandilian, C., Shah, N., & Sung, L. (2022) · Scientific Reports · doi:10.1038/s41598-022-06484-1 · Cited by 103 · see citing works
Guo, L. L., Pfohl, S. R., Fries, J., Posada, J., Fleming, S. L., Aftandilian, C., Shah, N., & Sung, L. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1735184 · Cited by 96 · see citing works
Callahan, A., McElfresh, D., Banda, J. M., Bunney, G., Char, D., Chen, J., Corbin, C. K., Dash, D., Downing, N. L., Jain, S. S., et al. (2024) · NEJM Catalyst · doi:10.1056/cat.24.0131 · Cited by 61 · see citing works
Advancing primary care with Artificial Intelligence and Machine Learning
Yang, Z., Silcox, C., Sendak, M., Rose, S., Rehkopf, D., Phillips, R., Peterson, L., Marino, M., Maier, J., Lin, S., et al. (2022) · Healthcare · doi:10.1016/j.hjdsi.2021.100594 · Cited by 38 · see citing works
Self-citationBuilding models, building capacity: A review of participatory machine learning for HIV prevention
Sendak, M., Young, M., Kim, J. Y., Hasan, A., Kelsey, C., O’Neal, C., Jagneaux, T., Wilbright, W., Couk, J., Lim, S., et al. (2025) · PLOS Global Public Health · doi:10.1371/journal.pgph.0003862 · Cited by 2 · see citing works
Self-citation
- Authors
- Sendak, M. P., Balu, S., & Schulman, K. A.
- Affiliations
- Duke University (US)
- Corresponding author
- Schulman, Kevin A.
- Venue
- Applied Clinical Informatics
- Publisher
- Georg Thieme Verlag KG
- Published
- 2017-07
- Volume
- 08
- Issue
- 03
- Pages
- 826-831
- ISSN
- 1869-0327 (ISSN-L)
- PMC
- PMC6220705
- OpenAlex
- W2743376227
- Cited by
- 45
- References
- 3
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 3 · 2023: 7 · 2022: 8 · 2021: 6 · 2020: 6 · 2019: 4 · 2018: 1 · 2017: 1
- FWCI
- 4.51
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Applied Clinical Informatics (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Duke Clinical Research Institute
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Keywords
- electronic health records, health information technology adoption, economies of scale, clinical analytics, chronic kidney disease, nephrology referral, high-risk patient identification, return on investment, health IT policy, multihospital systems, clinical workflow integration
- MeSH terms
- Humans, Medical Informatics, Referral and Consultation, Renal Insufficiency, Chronic, Workflow, Electronic Health Records, Inventions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
