The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
How Valid Are Medical Records and Patient Questionnaires for Physician Profiling and Health Services Research?: A Comparison With Direct Observation of Patient Visits
Abstract
OBJECTIVES: This study was designed to determine the optimal nonobservational method of measuring the delivery of outpatient medical services. METHODS: As part of a multimethod study of the content of primary care practice, research nurses directly observed consecutive patient visits to 138 practicing family physicians. Data on services delivered were collected using a direct observation checklist, medical record review, and patient exit questionnaires. For each medical service, the sensitivity, specificity, and Kappa statistic were calculated for medical record review and patient exit questionnaires compared with direct observation. Interrater reliability among eight research nurses was calculated using the Kappa statistic for a separate sample of videotaped visits and medical records. RESULTS: Visits by 4,454 patients were observed. Exit questionnaires were returned by 74% of patients. Research nurse interrater reliabilities were generally high. The specificity of both the medical record and the patient exit questionnaire was high for most services. The sensitivity of the medical record was low for measuring health habit counseling and moderate for physical examination, laboratory testing, and immunization. The patient exit questionnaire showed moderate to high sensitivity for health habit counseling and immunization and variable sensitivity for physical examination and laboratory services. CONCLUSIONS: The validity of the medical record and patient questionnaire for measuring delivery of different health services varied with the service. This report can be used to choose the optimal nonobservational method of measuring the delivery of specific ambulatory medical services for research and physician profiling and to interpret existing health services research studies using these common measures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
High agreement but low Kappa: I. the problems of two paradoxes
Feinstein, A. R., & Cicchetti, D. V. (1990) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(90)90158-l · Cited by 3,034
High agreement but low kappa: II. Resolving the paradoxes
Cicchetti, D. V., & Feinstein, A. R. (1990) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(90)90159-m · Cited by 1,838
The National Committee for Quality Assurance
Iglehart, J. K. (1996) · New England Journal of Medicine · doi:10.1056/nejm199609263351322 · Cited by 1,824
Misinterpretation and misuse of the kappa statistic
MACLURE, M., & WILLETT, W. C. (1987) · American Journal of Epidemiology · doi:10.1093/aje/126.2.161 · Cited by 808
A reason for visit classification for ambulatory care.
Schneider, D., Appleton, L., & McLemore, T. (1979) · PubMed · Cited by 795
The accuracy of Medicare's hospital claims data: progress has been made, but problems remain.
Fisher, E. S., Whaley, F. S., Krushat, W. M., Malenka, D. J., Fleming, C., Baron, J. A., & Hsia, D. C. (1992) · American Journal of Public Health · doi:10.2105/ajph.82.2.243 · Cited by 709
AGREEMENT BETWEEN QUESTIONNAIRE DATA AND MEDICAL RECORDS
HARLOW, S. D., & LINET, M. S. (1989) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a115129 · Cited by 609
Integration and generalization of kappas for multiple raters
Conger, A. J. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.2.322 · Cited by 545
Montaño, D. E., & Phillips, W. R. (1995) · American Journal of Public Health · doi:10.2105/ajph.85.6.795 · Cited by 349
Cited by
The Quality of Ambulatory Care Delivered to Children in the United States
Mangione-Smith, R., DeCristofaro, A. H., Setodji, C. M., Keesey, J., Klein, D. J., Adams, J. L., Schuster, M. A., & McGlynn, E. A. (2007) · New England Journal of Medicine · doi:10.1056/nejmsa064637 · Cited by 581
Continuity of Primary Care: To Whom Does It Matter and When?
Nutting, P. A. (2003) · The Annals of Family Medicine · doi:10.1370/afm.63 · Cited by 316
Tisnado, D. M., Adams, J. L., Liu, H., Damberg, C. L., Chen, W.-P., Hu, F. A., Carlisle, D. M., Mangione, C. M., & Kahn, K. L. (2006) · Medical Care · doi:10.1097/01.mlr.0000196952.15921.bf · Cited by 293
Racial and Ethnic Disparities in Smoking-Cessation Interventions
Cokkinides, V. E., Halpern, M. T., Barbeau, E. M., Ward, E., & Thun, M. J. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.02.003 · Cited by 280
Advanced statistics: Understanding Medical Record Review (MRR) Studies
Worster, A., & Haines, T. (2004) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2004.tb01433.x · Cited by 269
Metadata
- Authors
- Stange, K. C., Zyzanski, S. J., Smith, T. F., Kelly, R., Langa, D. M., Flocke, S. A., & Jaén, C. R.
- Venue
- Medical Care
- Published
- 1998-06
- Volume
- 36
- Issue
- 6
- Pages
- 851-867
- ISSN
- 0025-7079 (ISSN-L)
- OpenAlex
- W2050157657
- Cited by
- 313
- References
- 45
- FWCI
- 32.72
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How Valid Are Medical Records and Patient Questionnaires for Physician Profiling and Health Services Research?: A Comparison With Direct Observation of Patient Visits. https://app.cochat.ai/papers/10.1097/00005650-199806000-00009
- The paper
- Stange, K. C., Zyzanski, S. J., Smith, T. F., Kelly, R., Langa, D. M., Flocke, S. A., & Jaén, C. R. (1998). How Valid Are Medical Records and Patient Questionnaires for Physician Profiling and Health Services Research?: A Comparison With Direct Observation of Patient Visits. Medical Care. https://doi.org/10.1097/00005650-199806000-00009
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.