Effect of electronic health records in ambulatory care: retrospective, serial, cross sectional study
Open accessAbstract
OBJECTIVE: To evaluate the effect of implementing comprehensive, integrated electronic health record systems on use and quality of ambulatory care DESIGN: Retrospective, serial, cross sectional study. SETTING: Colorado and Northwest regions of Kaiser Permanente, a US integrated healthcare delivery system. POPULATION: 367,795 members in the Colorado region and 449,728 members in the Northwest region. INTERVENTION: Implementation of electronic health record systems. MAIN OUTCOME MEASURES: Total number of office visits and use of primary care, specialty care, clinical laboratory, radiology services, and telephone contact. Health Plan Employer Data and Information Set to assess quality. RESULTS: Two years after electronic health records were fully implemented, age adjusted rates of office visits fell by 9% in both regions. Age adjusted primary care visits decreased by 11% in both regions and specialty care visits decreased by 5% in Colorado and 6% in the Northwest. All these decreases were significant (P < 0.0001). The percentage of members making > or = 3 visits a year decreased by 10% in Colorado and 11% in the Northwest, and the percentage of members with < or = 2 visits a year increased. In the Northwest, scheduled telephone contact increased from a baseline of 1.26 per member per year to 2.09 after two years. Use of clinical laboratory and radiology services did not change conclusively. Intermediate measures of quality of health care remained unchanged or improved slightly. CONCLUSIONS: Readily available, comprehensive, integrated clinical information reduced use of ambulatory care while maintaining quality and allowed doctors to replace some office visits with telephone contacts. Shifting patterns of use suggest reduced numbers of ambulatory care visits that are inappropriate or marginally productive.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In that Kaiser analysis, because of the large numbers, all differences were statistically significant.
“Because of the large numbers, all differences were statistically significant.”
From The Empirical Foundations of Telemedicine Interventions in Primary Care (2016) · cites this paper as [81]
The Statistical analysis section (p. 2) says the large sample size justified using a z test; it does not say that size made every difference significant. The Use of ambulatory care results (p. 3) report significant decreases in office visits, but the paper does not establish statistical significance for all differences it examined. (LLM, conf=0.87)
Experimental — accuracy not yet measuredIn that Kaiser analysis, specialty care declined by 5% in Colorado and 6% in the Northwest.
“Specialty care declined by 5% in Colorado and 6% in the Northwest.”
From The Empirical Foundations of Telemedicine Interventions in Primary Care (2016) · cites this paper as [81]
In a retrospective analysis at Kaiser Permanente Colorado (n=376,795) and Northwest (n=449,728), age-adjusted office visits decreased by 9% and primary care visits declined by 11% in both regions.
“Overall, age-adjusted office visits decreased by 9%, whereas primary care visits declined by 11% in both regions.”
From The Empirical Foundations of Telemedicine Interventions in Primary Care (2016) · cites this paper as [81]
In that Kaiser analysis, use of laboratory and radiology services did not change conclusively, and intermediate measures of quality remained unchanged or improved slightly.
“Use of laboratory and radiology services did not change conclusively, and intermediate measures of quality remained unchanged or improved slightly.”
From The Empirical Foundations of Telemedicine Interventions in Primary Care (2016) · cites this paper as [81]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The quality of health care delivered to adults in the United States
McGlynn, E. A., Asch, S. M., Adams, J., Keesey, J., Hicks, J., DeCristofaro, A., & Kerr, E. A. (2003) · New England Journal of Medicine · doi:10.1056/nejmsa022615 · Cited by 5,079 · see citing works
The National Committee for Quality Assurance
Iglehart, J. K. (1996) · New England Journal of Medicine · doi:10.1056/nejm199609263351322 · Cited by 1,839 · see citing works
The Computer-Based Patient Record: An Essential Technology for Health Care
(1992) · Annals of Internal Medicine · doi:10.7326/0003-4819-117-11-976_1 · Cited by 999 · see citing works
The Urgent Need to Improve Health Care Quality
Chassin, M. R., & Galvin, R. W. (1998) · DigitalGeorgetown (Georgetown University Library) · doi:10.1001/jama.280.11.1000 · Cited by 681 · see citing works
Physician inpatient order writing on microcomputer workstations. Effects on resource utilization
Tierney, W. M. (1993) · JAMA · doi:10.1001/jama.269.3.379 · Cited by 561 · see citing works
Effects of Computerized Physician Order Entry on Prescribing Practices
Teich, J. M., Merchia, P., Schmiz, J. L., Kuperman, G. J., Spurr, C. D., & Bates, D. W. (2000) · Archives of Internal Medicine · doi:10.1001/archinte.160.18.2741 · Cited by 475 · see citing works
Position Paper: A Proposal for Electronic Medical Records in U.S. Primary Care
Bates, D., Ebell, M., Gotlieb, E., Zapp, J., & Mullins, H. C. (2003) · J. Am. Medical Informatics Assoc. · doi:10.1197/jamia.m1097 · Cited by 411 · see citing works
National Ambulatory Medical Care Survey: 2007 summary.
Hsiao, C., Cherry, D. K., Beatty, P. C., & Rechtsteiner, E. A. (2010) · PubMed · Cited by 324 · see citing works
National Ambulatory Medical Care Survey: 2002 summary.
Woodwell, D. A., & Cherry, D. K. (2004) · PubMed · Cited by 314 · see citing works
Hersh, W. (2002) · JAMA · doi:10.1001/jama.288.16.1955 · Cited by 193 · see citing works
Hidden Barriers to Improvement in the Quality of Care
McNeil, B. J. (2001) · New England Journal of Medicine · doi:10.1056/nejmsa011810 · Cited by 157 · see citing works
The Misinformation Era: The Fall of the Medical Record
Burnum, J. F. (1989) · Annals of Internal Medicine · doi:10.7326/0003-4819-110-6-482 · Cited by 129 · see citing works
Chassin, & Galvin (1998) · JAMA · Cited by 102 · see citing works
Assessment of Decision Support for Blood Test Ordering in Primary Care
van Wijk, M. A. M., van der Lei, J., Mosseveld, M., Bohnen, A. M., & van Bemmel, J. H. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-4-200102200-00010 · Cited by 98 · see citing works
Gadd, C. S., & Penrod, L. E. (2001) · PubMed · Cited by 59 · see citing works
Resident Documentation Discrepancies in a Neonatal Intensive Care Unit
Carroll, A. E., Tarczy-Hornoch, P. Z., O’Reilly, E. B., et al. (2003) · PEDIATRICS · doi:10.1542/peds.111.5.976 · Cited by 52 · see citing works
Measuring the success of electronic medical record implementation using electronic and survey data.
Keshavjee, K., Troyan, S., Holbrook, A. M., & VanderMolen, D. (2001) · Europe PMC (PubMed Central) · Cited by 44 · see citing works
Acceptance and performance by clinicians using an ambulatory electronic medical record in an HMO.
Ma, K. (1995) · PubMed · Cited by 34 · see citing works
Increasing the acceptance of clinical information systems.
Anderson, J. D. (1999) · PubMed · Cited by 32 · see citing works
Patterns of decline among inpatient procedures.
Duffy, S., & Farley, D. E. (1996) · PubMed · Cited by 31 · see citing works
Evaluating the Impact of a Computerized Ambulatory Record
Bates, D. W., Studer, J., Reilly, C. A., et al. (2000) · PubMed Central · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [4]4. Bates DW, Ebell M, Gotlieb E, Zapp J, Mullins HC. A proposal for electronic medical records in US primary care. J Am Med Inform Assoc 2003;10:1-10.
- [21]21. Bates DW, Ebell M, Gotlieb E, Zapp J, Mullins HC. A proposal for electronic medical records in US primary care. J Am Med Inform Assoc 2003;10:1-10.
Cited by
Sorted by citations
Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., Morton, S. C., & Shekelle, P. G. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-10-200605160-00125 · Cited by 3,159 · see citing works
e-Health technologies show promise in developing countries
Blaya, J. A., Fraser, H. S. F., & Holt, B. (2010) · Health Affairs · doi:10.1377/hlthaff.2009.0894 · Cited by 581 · see citing works
Costs and benefits of health information technology.
Shekelle, P. G., Morton, S. C., & Keeler, E. B. (2006) · doi:10.23970/ahrqepcerta132 · Cited by 501 · see citing works
The Empirical Foundations of Telemedicine Interventions in Primary Care
Bashshur, R. L., Howell, J. D., Krupinski, E. A., Harms, K. M., Bashshur, N., & Doarn, C. R. (2016) · Telemedicine and e-Health · doi:10.1089/tmj.2016.0045 · Cited by 370 · see citing works
Kaiser Permanente's experience of implementing an electronic medical record: a qualitative study
Scott, J. T., Rundall, T. G., Vogt, T. M., & Hsu, J. (2005) · BMJ · doi:10.1136/bmj.38638.497477.68 · Cited by 266 · see citing works
Pollak, V. E., & Lorch, J. A. (2007) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-7-38 · Cited by 40 · see citing works
- Authors
- Garrido, T., Jamieson, L., Zhou, Y., Wiesenthal, A., & Liang, L.
- Affiliations
- Kaiser Permanente (US)
- Corresponding author
- Garrido, Terhilda
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2005-03-10
- Volume
- 330
- Issue
- 7491
- Pages
- 581
- ISSN
- 1756-1833 (ISSN-L)
- 0959-8138
- PMC
- PMC554034
- OpenAlex
- W2139759021
- Semantic Scholar
- 5ec48dea62cc6f51d070bc77a0f208614b4e6709
- Cited by
- 152
- References
- 21
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 2 · 2022: 1 · 2021: 2 · 2020: 5 · 2019: 1 · 2018: 5 · 2017: 5 · 2016: 5 · 2015: 7 · 2014: 10 · 2013: 9 · 2012: 14
- FWCI
- 22.35
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Kaiser Permanente
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › General Health Professions › Primary Care and Health Outcomes
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Healthcare Systems and Technology
- Keywords
- electronic health records, ambulatory care utilization, office visits, primary care visits, telephone contact, quality of care, Kaiser Permanente, integrated care delivery, clinical laboratory services, radiology services
- MeSH terms
- Age Factors, Ambulatory Care, Colorado, Cross-Sectional Studies, Humans, Quality of Health Care, Radiology Department, Hospital, Retrospective Studies, Medical Records Systems, Computerized, Remote Consultation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
