The DOI matches its Crossref record; 28 of the 33 references it cites were found in a registry or an index; 12 of the 17 sources its claims cite were checked for support.
Effect of an Electronic Health Record-Based Intervention on Documentation Practices
Open accessAbstract
Abstract Background Documentation burden is one of the largest contributors to physician burnout. Evaluation and Management (E&M) coding changes were implemented in 2021 to alleviate documentation burden. Objectives We used this opportunity to develop documentation best practices, implement new electronic health record (EHR) tools, and study the potential impact on provider experiences with documentation related to these 2021 E&M changes, documentation length, and time spent documenting at an academic medical center. Methods Five actionable best practices, developed through a consensus-driven, multidisciplinary approach in November 2020, led to the creation of two new ambulatory note templates, one for E&M visits (implemented in January 2021) and another for preventative visits (implemented in May 2021). As part of a quality-improvement initiative at nine faculty primary care clinics, surveys were developed utilizing a 5-point Likert scale to assess provider perceptions and deidentified EHR metadata (Signal, Epic Systems) were analyzed to measure changes in EHR use metrics between a pre-E&M changes timeframe (August 2020–December 2020) and a post-E&M change timeframe (August 2021–December 2021). A subgroup analysis was conducted comparing EHR use metrics among note template utilizers versus nonutilizers. Any provider who used one of the note templates at least once was categorized as a utilizer. Results Between January 2021 and December 2021, the adoption of the E&M visit template was 31,480 instances among 120 unique ambulatory providers, and adoption of the preventative visit template was 1,464 instances among 22 unique ambulatory providers. Survey response rate among faculty primary care providers was 82% (88/107): 55% (48/88) believed the 2021 E&M changes provided an opportunity to reduce documentation burden, and 28% reported favorable satisfaction with time spent documenting. Among providers who reported using one or both of the new note templates, 81% (35/43) of survey respondents reported favorable satisfaction with new note templates. EHR use metric analyses revealed a small, yet significant reduction in time in notes per appointment (p = 0.004) with no significant change in documentation length of notes (p = 0.45). Note template utilization was associated with a statistically significant reduction in documentation length (p = 0.034). Conclusion This study shows modest progress in improving EHR use measures of documentation length and time spent documenting following the 2021 E&M changes, but without great improvement in perceived documentation burden. Additional tools are needed to reduce documentation burden and further research is needed to understand the impact of these interventions.
Overview
CoChat Summary
TL;DR
This quality-improvement study at an academic medical center developed documentation best practices and two new Epic note templates in response to the 2021 E&M coding changes, then assessed provider perceptions and EHR use metrics. It found a small but significant reduction in time in notes per appointment and no significant change in note length overall, with template utilization associated with reduced note length, but little improvement in perceived documentation burden.
Key findings
- Organization-wide adoption between January and December 2021 reached 31,480 instances of the E&M visit template among 120 unique providers and 1,464 instances of the preventative visit template among 22 unique providers.
- Survey response rate among faculty primary care providers was 82% (88/107); 55% (48/88) believed the 2021 E&M changes provided an opportunity to reduce documentation burden, and 28% (25/88) reported satisfaction with time spent documenting. Among template users, 81% (35/43) reported favorable satisfaction with the E&M template and 89% (16/18) with the preventative template.
- EHR use metrics (64 providers in final analysis) showed a small but significant reduction in time in notes per appointment (−0.84 minutes, 7.8%, p = 0.004) and no significant change in documentation length (−54.24 characters, p = 0.4476).
- Subgroup analysis found template utilization was significantly associated with reduced documentation length (utilizers −274.026 vs. nonutilizers +69.057 characters, p = 0.0337), but not with time per appointment (p = 0.4726).
Methods
- A 22-member multidisciplinary workgroup (14 providers from primary care, gastroenterology, and dermatology) met roughly weekly from October 2020 to May 2021 and developed five documentation best practices and five structural design principles; two note templates were built in Epic, implemented in January 2021 (E&M) and May 2021 (preventative), alongside a level-of-service calculator update, encounter summary updates, and voluntary multimodal training.
- A four-question, 5-point Likert survey was emailed in December 2021 to all 107 providers in the academic primary care division (nonanonymized, voluntary response sampling); completion was defined as any response to one or more questions, and items were scored as proportions per response category.
- Deidentified EHR metadata (Signal, Epic Systems) were analyzed for faculty primary care providers comparing pre-E&M (August–December 2020) and post-E&M (August–December 2021) timeframes using a one-sample t-test, with a utilizer-vs-nonutilizer subgroup analysis using Welch's two-sample t-test; α = 0.05, analyses in R.
Limitations
- Stated by the paper: the study focused on primary care, while subspecialties also experience documentation burden and impacts likely vary; surveys were nonanonymized and lacked formal criterion and construct validation; no analysis compared training module completers versus noncompleters; the utilization measure missed providers who copied and edited templates into personal versions; the two templates were implemented at different times (January vs. May), likely affecting utilization; and EHR metadata have known limitations, especially for time-based approximations, in a natural experiment without randomized design.
- Evident from the text: the survey was distributed only to primary care faculty, so perception findings may not generalize to the broader organization-wide adopter population; the small magnitude of the significant time reduction (−0.84 minutes) limits its practical significance.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
10 Valid · 0 Not valid · 1 Not checked · 45 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Evaluation and Management (E&M) coding changes were implemented in 2021 to alleviate documentation burden.”
- Also stated in: § Background and Significance · plain
“To directly address the continuing problem of documentation burden, the American Medical Association (AMA) and the Centers for Medicare and Medicaid Services (CMS) implemented the 2021 billing Evaluation & Management (E&M) rules.”
§ Effect of an Electronic Health Record-Based Intervention on Documentation Practices > Abstract
What the sources say
- Supportedcites [13] Flamm A, Bridges A, Siegel D M. E/M coding in 2021: the times (and more) are a-changin'. Cutis. 2021;107(06):301–325. doi: 10.12788/cutis.0270. [[DOI](https://doi.org/10.12788/cutis.0270)] [[PubMed](https://pubmed.ncbi.nlm.nih.gov/34314312/)] [[Google Scholar](https://scholar.google.com/scholar_lookup?journal=Cutis&title=E/M%20coding%20in%202021:%20the%20times%20(and%20more)%20are%20a-changin'&author=A%20Flamm&author=A%20Bridges&author=D%20M%20Siegel&volume=107&issue=06&publication_year=2021&pages=301-325&pmid=34314312&doi=10.12788/cutis.0270&)] source
The opening section says the changes took effect on January 1, 2021, in part to decrease administrative burden. The passage continued on p. 325 explicitly describes an intended decrease in documentation burden. (LLM, conf=0.94)
- Partly supportedcites [14] CPT ® Evaluation and Management (E/M) Office or Other Outpatient (99202–99215) and Prolonged Services (99354, 99355, 99356, 99417) Code and Guideline Changes. American Medical Association;2021. Accessed July 18, 2024 at:<https://www.ama-assn.org/system/files/2019-06/cpt-office-prolonged-svs-code-changes.pdf> sourceExperimental — accuracy not yet measured
The paper’s discussion of the proposed 2019 Medicare Physician Fee Schedule says E&M guideline reform was intended to reduce regulatory documentation burden. In “Will Documentation Burden Decrease as a Result of This Proposal?” (page unavailable), the authors say the proposal had yet to be finalized. The paper therefore supports the stated aim, but not the claim that changes were implemented in 2021. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites [15] Basch P, Smith J RL. CMS payment policy, E&M guideline reform, and the prospect of electronic health record optimization. Appl Clin Inform. 2018;9(04):914–918. doi: 10.1055/s-0038-1676337. [[DOI](https://doi.org/10.1055/s-0038-1676337)] [[PMC free article](/articles/PMC6306276/)] [[PubMed](https://pubmed.ncbi.nlm.nih.gov/30586672/)] [[Google Scholar](https://scholar.google.com/scholar_lookup?journal=Appl%20Clin%20Inform&title=CMS%20payment%20policy,%20E&M%20guideline%20reform,%20and%20the%20prospect%20of%20electronic%20health%20record%20optimization&author=P%20Basch&author=J%20RL%20Smith&volum source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Background and Significance · plain
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Documentation burden is one of the largest contributors to physician burnout.”
- Also stated in: § Background and Significance · plain
“Physician burnout attributed to the electronic health record (EHR) and increased documentation burden due to a combination of clinical documentation, regulatory, and billing requirements, is plaguing the U.S. health care system.”
§ Effect of an Electronic Health Record-Based Intervention on Documentation Practices > Abstract
What the sources say
- Not checked — no full text of the sourcecites [1] Shanafelt T D, Dyrbye L N, Sinsky C et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc. 2016;91(07):836–848. doi: 10.1016/j.mayocp.2016.05.007. [[DOI](https://doi.org/10.1016/j.mayocp.2016.05.007)] [[PubMed](https://pubmed.ncbi.nlm.nih.gov/27313121/)] [[Google Scholar](https://scholar.google.com/scholar_lookup?journal=Mayo%20Clin%20Proc&title=Relationship%20between%20clerical%20burden%20and%20characteristics%20of%20the%20electronic%20environment%20with%20physician%20burnout%20and%20p source
- Partly supportedcites [2] Tajirian T, Stergiopoulos V, Strudwick G et al. The influence of electronic health record use on physician burnout: cross-sectional survey. J Med Internet Res. 2020;22(07):e19274. doi: 10.2196/19274. [[DOI](https://doi.org/10.2196/19274)] [[PMC free article](/articles/PMC7392132/)] [[PubMed](https://pubmed.ncbi.nlm.nih.gov/32673234/)] [[Google Scholar](https://scholar.google.com/scholar_lookup?journal=J%20Med%20Internet%20Res&title=The%20influence%20of%20electronic%20health%20record%20use%20on%20physician%20burnout:%20cross-sectional%20survey&author=T%20Tajirian&author=V%20Stergiopoulos&author sourceExperimental — accuracy not yet measured
The Overview of Physician Burnout (page unavailable) says excessive EHR data entry adds to physicians’ cognitive load, and the Abstract (page unavailable) identifies EHR use as a perceived contributor to burnout and recommends reducing its administrative burdens. The paper does not establish that documentation burden is one of the largest contributors relative to other causes. (LLM, conf=0.85)
- Not checked — no full text of the sourcecites [3] Gardner R L, Cooper E, Haskell J et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc. 2019;26(02):106–114. doi: 10.1093/jamia/ocy145. [[DOI](https://doi.org/10.1093/jamia/ocy145)] [[PMC free article](/articles/PMC7647171/)] [[PubMed](https://pubmed.ncbi.nlm.nih.gov/30517663/)] [[Google Scholar](https://scholar.google.com/scholar_lookup?journal=J%20Am%20Med%20Inform%20Assoc&title=Physician%20stress%20and%20burnout:%20the%20impact%20of%20health%20information%20technology&author=R%20L%20Gardner&author=E%20Cooper&author=J%20Haskell&volume=26&issu source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Background and Significance · plain
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Studies following the 2021 implementation identified a change in the billing codes used for visits without a clinically significant associated decrease in note length or time spent in the EHR.”
§ Effect of an Electronic Health Record-Based Intervention on Documentation Practices > Background and Significance
What the sources say
- Partly supportedcites [18] Apathy N C, Hare A J, Fendrich S, Cross D A. Early changes in billing and notes after evaluation and management guideline change. Ann Intern Med. 2022;175(04):499–504. doi: 10.7326/M21-4402. [[DOI](https://doi.org/10.7326/M21-4402)] [[PubMed](https://pubmed.ncbi.nlm.nih.gov/35188791/)] [[Google Scholar](https://scholar.google.com/scholar_lookup?journal=Ann%20Intern%20Med&title=Early%20changes%20in%20billing%20and%20notes%20after%20evaluation%20and%20management%20guideline%20change&author=N%20C%20Apathy&author=A%20J%20Hare&author=S%20Fendrich&author=D%20A%20Cross&volume=175&issue=04&publication sourceExperimental — accuracy not yet measured
The Discussion (page not provided) reports that reductions in EHR documentation time after the 2021 coding-requirement changes were not clinically meaningful. The paper does not report a change in the billing codes actually used for visits or measure note length; its time finding concerns documentation, not total time spent in the EHR. (LLM, conf=0.88)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“however, the magnitude of change was small.”
§ Effect of an Electronic Health Record-Based Intervention on Documentation Practices > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“a statistically significant reduction in time in notes per appointment”
§ Effect of an Electronic Health Record-Based Intervention on Documentation Practices > Discussion
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
24 more claims
Create a free account to see all 56 claims and every source verdict.
- Probably not a substantial claim. 24
- It may state two separate claims as one. 6
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effect of an Electronic Health Record-Based Intervention on Documentation Practices > Discussion
“The two note templates were implemented at different times (one in January and another in May) and this likely impacted note template utilization for 2020.”
§ Effect of an Electronic Health Record-Based Intervention on Documentation Practices > Methods > EHR Tool Development
“The E&M visit note template was implemented in January 2021, and a preventative visit note template was implemented in May 2021.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“38% (37/103 barriers) for the preventative note”
§ Effect of an Electronic Health Record-Based Intervention on Documentation Practices > Results > Primary Care Metrics
Reported 38 · recomputed 35.92233
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effect of an Electronic Health Record-Based Intervention on Documentation Practices > Discussion
“the vast majority (72%) remain dissatisfied with the time spent documenting in the EHR.”
§ Effect of an Electronic Health Record-Based Intervention on Documentation Practices > Results > Primary Care Metrics
“| Satisfaction with time spent documenting ( *n* = 88) | 5% (4/88) | 24% (21/88) | 17% (15/88) | 35% (31/88) | 19% (17/88) |”
Citations
Cites
Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties
Sinsky, C., Colligan, L., Li, L., Prgomet, M., Reynolds, S., Goeders, L., Westbrook, J., Tutty, M., & Blike, G. (2016) · Annals of Internal Medicine · doi:10.7326/m16-0961 · Cited by 1,494 · cited 1×
“” 8,9 Clinical notes in the United States are nearly four times longer on average than those in other countries, thought to be related to additional compliance and reimbursement documentation that is necessary for clinical notes in the United States.”
Shanafelt, T. D., Dyrbye, L. N., Sinsky, C., Hasan, O., Satele, D., Sloan, J., & West, C. P. (2016) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2016.05.007 · Cited by 1,086
Physician stress and burnout: the impact of health information technology
Gardner, R. L., Cooper, E., Haskell, J., Harris, D. A., Poplau, S., Kroth, P. J., & Linzer, M. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy145 · Cited by 566
Physician Burnout in the Electronic Health Record Era: Are We Ignoring the Real Cause?
Downing, N. L., Bates, D. W., & Longhurst, C. A. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0139 · Cited by 418 · cited 1×
“10,11 These note templates have the potential to standardize documentation practices and alleviate documentation burden, but the variability in use and the extensive individual customization options can be a challenge.”
Walker, J., Leveille, S., Bell, S., Chimowitz, H., Dong, Z., Elmore, J. G., Fernandez, L., Fossa, A., Gerard, M., Fitzgerald, P., et al. (2019) · Journal of Medical Internet Research · doi:10.2196/13876 · Cited by 309
The influence of electronic health record use on physician burnout: cross-sectional survey
Tajirian, T., Stergiopoulos, V., Strudwick, G., Sequeira, L., Sanches, M., Kemp, J., Ramamoorthi, K., Zhang, T., & Jankowicz, D. (2020) · Journal of Medical Internet Research · doi:10.2196/19274 · Cited by 241
Documentation burden in nursing and its role in clinician burnout syndrome
Gesner, E., Dykes, P. C., Zhang, L., & Gazarian, P. (2022) · Applied Clinical Informatics · doi:10.1055/s-0042-1757157 · Cited by 118
Medical Documentation Burden Among US Office-Based Physicians in 2019: A National Study
Gaffney, A., Woolhandler, S., Cai, C., Bor, D., Himmelstein, J., McCormick, D., & Himmelstein, D. U. (2022) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2022.0372 · Cited by 118
Cusack, C. M., Hripcsak, G., Bloomrosen, M., Rosenbloom, S. T., Weaver, C. A., Wright, A., Vawdrey, D. K., Walker, J., & Mamykina, L. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001093 · Cited by 103
Length and redundancy of outpatient progress notes across a decade at an academic medical center
Rule, A., Bedrick, S., Chiang, M. F., & Hribar, M. R. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.15334 · Cited by 90
Cited by
Doherty, R., Kazley, A. S., Karp, E. L., & Ferrand, J. L. (2025) · Applied Clinical Informatics · doi:10.1055/a-2751-1896 · Cited by 10
EHR Use in Inpatient Physicians: Patterns and Predictors
Safian, G., Tio, K., Tang, K., Arora, S., & Jariwala, S. P. (2025) · Applied Clinical Informatics · doi:10.1055/a-2615-4085 · Cited by 1
Typing Proficiency among Physicians in Internal Medicine: A Pilot Study of Speed and Performance
Bastardot, F., Kraege, V., Castioni, J., Petter, A., Bates, D. W., & Garnier, A. (2025) · Applied Clinical Informatics · doi:10.1055/a-2620-3147 · Cited by 1
SAIMSARA (2026) · SAIMSARA Journal · doi:10.62487/saimsara40f2f857
Metadata
- Authors
- Shah, S., Bedgood, M., Devon-Sand, A., Dolphin-Dempsey, C., Cherukuri, V., Weng, K., Lin, S., & Sharp, C.
- Affiliations
- Stanford Medicine (US)
- California Department of Health Care Services (US)
- Stanford Health Care (US)
- Corresponding author
- Shah, Shreya
- Venue
- Applied Clinical Informatics
- Publisher
- Georg Thieme Verlag KG
- Published
- 2024-07-17
- Volume
- 15
- Issue
- 04
- Pages
- 771-777
- ISSN
- 1869-0327 (ISSN-L)
- PMC
- PMC11424194
- OpenAlex
- W4400724463
- Cited by
- 4
- References
- 33
- Citations by year
- 2026: 1 · 2025: 3
- FWCI
- 1.22
- Type
- journal article
- Language
- en
- Open access
- green
- pmc.ncbi.nlm.nih.gov
- Locations
- Applied Clinical Informatics (publishedVersion)
- PubMed Central (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Health Sciences › Nursing › Research and Theory › Nursing Diagnosis and Documentation
- Keywords
- documentation burden, electronic health records, note templates, physician burnout, documentation practices, outpatient care, primary care, documentation time, quality improvement, provider satisfaction, academic medical centers, preventive visits, faculty physicians, Epic EHR
- MeSH terms
- Documentation, Humans, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Effect of an Electronic Health Record-Based Intervention on Documentation Practices. https://app.cochat.ai/papers/10.1055/a-2367-8564
- The paper
- Shah, S., Bedgood, M., Devon-Sand, A., Dolphin-Dempsey, C., Cherukuri, V., Weng, K., Lin, S., & Sharp, C. (2024). Effect of an Electronic Health Record-Based Intervention on Documentation Practices. Applied Clinical Informatics. https://doi.org/10.1055/a-2367-8564
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.