Assessing electronic note quality using the physician documentation quality instrument (PDQI-9)
Open accessAbstract
Summary: Objective: To refine the Physician Documentation Quality Instrument (PDQI) and test the validity and reliability of the 9-item version (PDQI-9). Methods: Three sets each of admission notes, progress notes and discharge summaries were evaluated by two groups of physicians using the PDQI-9 and an overall general assessment: one gold standard group consisting of program or assistant program directors (n = 7), and the other of attending physicians or chief residents (n = 24). The main measures were criterion-related validity (correlation coefficients between Total PDQI-9 scores and 1-item General Impression scores for each note), discriminant validity (comparison of PDQI-9 scores on notes rated as best and worst using 1-item General Impression score), internal consistency reliability (Cronbach’s alpha), and inter-rater reliability (intraclass correlation coefficient (ICC)). Results: The results were criterion-related validity (r = –0.678 to 0.856), discriminant validity (best versus worst note, t = 9.3, p = 0.003), internal consistency reliability (Cronbach’s alphas = 0.87–0.94), and inter-rater reliability (ICC = 0.83, CI = 0.72–0.91). Conclusion: The results support the criterion-related and discriminant validity, internal consistency reliability, and inter-rater reliability of the PDQI-9 for rating the quality of electronic physician notes. Tools for assessing note redundancy are required to complement use of PDQI-9. Trials of the PDQI-9 at other institutions, of different size, using different EHRs, and incorporating additional physician specialties and notes of other healthcare providers are needed to confirm its generaliz-ability.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
When compared to previous literature, AI-driven documentation varied between 'bad (29.7)' and 'excellent (35.6)'.
“When compared to previous literature AI-driven documentation varied between “bad (29.7)” and “excellent (35.6)” highlighting the variability in documentation produced by AI systems [23, 25, 28].”
From Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review (2025) · cites this paper as [28]
The Results (Discriminant validity; page unavailable) report variation in the quality of electronic notes, but the scores are 26.2 for a note rated Terrible or Bad and 36.6 for one rated Good or Excellent—not 29.7 and 35.6. The study assessed physician documentation, not AI-driven documentation, so it does not support that part of the claim. (LLM, conf=0.86)
Experimental — accuracy not yet measuredEach summary was independently evaluated by an MD using the PDQI-10, a validated measure adapted specifically for CAISs.
“Each summary was independently evaluated by an MD using the PDQI-10,29 47 a validated measure adapted specifically for CAISs.”
From Clinical AI Scribes in primary care: accuracy, error severity and implications for clinical practice (2025) · cites this paper as [47]
The Methods §3.1 and Discussion §5 describe a nine-item PDQI and preliminary evidence of its validity for assessing clinical notes, including discharge summaries. This source does not establish the validity of a PDQI-10 adapted for CAISs, or document the citing study’s MD evaluations. Page numbers were not available in our copy. (LLM, conf=0.88)
Experimental — accuracy not yet measuredClinical note quality was assessed using the validated 9-item Physician Documentation Quality Instrument (PDQI-9), which assigns a quality score across 9 measures.
“Clinical note quality was assessed using the 9-item Physician Documentation Quality Instrument 9-item version (PDQI-9).17 This validated, objective note assessment tool assigns a quality score (range: 9-45) across 9 measures scored on a 5-point Likert scale: up-to-date, accurate, thorough, useful, organized, comprehensible, succinct, synthesized, and consistent.”
From Evaluating the Quality and Safety of Ambient Digital Scribe Platforms Using Simulated Ambulatory Encounters (2025) · cites this paper as [17]
Neither the PDQI-10 nor the Impact Score currently has a validated threshold for clinical acceptability.
“However, neither metric currently has a validated threshold for clinical acceptability,29 47”
From Clinical AI Scribes in primary care: accuracy, error severity and implications for clinical practice (2025) · cites this paper as [47]
The Methods (§3.1, §3.4) describe the PDQI-9 and tests of its validity, but do not address a threshold for clinical acceptability. Searches for a validated threshold, PDQI-10, and Impact Score found no passage addressing the claim in this paper. (LLM, conf=0.77)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 40 · Cited by 182
- Authors
- Bakken, S., Wrenn, J. O., Siegler, E. L., & Stetson, P. D.
- Affiliations
- Columbia University (US)
- Vanderbilt University (US)
- Weill Cornell Medicine (US)
- Venue
- Applied Clinical Informatics
- Publisher
- Georg Thieme Verlag KG
- Published
- 2012
- Volume
- 03
- Issue
- 02
- Pages
- 164-174
- ISSN
- 1869-0327 (ISSN-L)
- PMC
- PMC3347480
- OpenAlex
- W2139056371
- Cited by
- 182
- References
- 40
- Citations by year
- 2026: 45 · 2025: 38 · 2024: 15 · 2023: 8 · 2022: 10 · 2021: 11 · 2020: 8 · 2019: 6 · 2018: 6 · 2017: 7 · 2016: 9 · 2015: 4 · 2014: 9 · 2013: 4 · 2012: 2
- FWCI
- 9.85
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Applied Clinical Informatics (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- U.S. National Library of Medicine (K22LM008805)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Nursing › Issues, ethics and legal aspects › Nursing Diagnosis and Documentation
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Keywords
- electronic health records, admission notes, progress notes, discharge summaries, validity, reliability statistics, inter-rater reliability, internal consistency, criterion validity, discriminant validity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
