The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Inter-rater reliability of nursing home quality indicators in the U.S
Open accessAbstract
BACKGROUND: In the US, Quality Indicators (QI's) profiling and comparing the performance of hospitals, health plans, nursing homes and physicians are routinely published for consumer review. We report the results of the largest study of inter-rater reliability done on nursing home assessments which generate the data used to derive publicly reported nursing home quality indicators. METHODS: We sampled nursing homes in 6 states, selecting up to 30 residents per facility who were observed and assessed by research nurses on 100 clinical assessment elements contained in the Minimum Data Set (MDS) and compared these with the most recent assessment in the record done by facility nurses. Kappa statistics were generated for all data items and derived for 22 QI's over the entire sample and for each facility. Finally, facilities with many QI's with poor Kappa levels were compared to those with many QI's with excellent Kappa levels on selected characteristics. RESULTS: A total of 462 facilities in 6 states were approached and 219 agreed to participate, yielding a response rate of 47.4%. A total of 5758 residents were included in the inter-rater reliability analyses, around 27.5 per facility. Patients resembled the traditional nursing home resident, only 43.9% were continent of urine and only 25.2% were rated as likely to be discharged within the next 30 days. Results of resident level comparative analyses reveal high inter-rater reliability levels (most items >.75). Using the research nurses as the "gold standard", we compared composite quality indicators based on their ratings with those based on facility nurses. All but two QI's have adequate Kappa levels and 4 QI's have average Kappa values in excess of.80. We found that 16% of participating facilities performed poorly (Kappa <.4) on more than 6 of the 22 QI's while 18% of facilities performed well (Kappa >.75) on 12 or more QI's. No facility characteristics were related to reliability of the data on which Qis are based. CONCLUSION: While a few QI's being used for public reporting have limited reliability as measured in US nursing homes today, the vast majority of QI's are measured reliably across the majority of nursing facilities. Although information about the average facility is reliable, how the public can identify those facilities whose data can be trusted and whose cannot remains a challenge.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
High agreement but low Kappa: I. the problems of two paradoxes
Feinstein, A. R., & Cicchetti, D. V. (1990) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(90)90158-l · Cited by 3,034
Effect of the National Resident Assessment Instrument on Selected Health Conditions and Problems
Fries, B. E., Hawes, C., Morris, J. N., Phillips, C. D., Mor, V., & Park, P. S. (1997) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.1997.tb02972.x · Cited by 161
Self-citation
Cited by
Hirdes, J. P., Ljunggren, G., Morris, J. N., Frijters, D. H., Finne Soveri, H., Gray, L., Björkgren, M., & Gilgen, R. (2008) · BMC Health Services Research · doi:10.1186/1472-6963-8-277 · Cited by 665
Self-citationDriven to tiers: socioeconomic and racial disparities in the quality of nursing home care
MOR, V., ZINN, J., ANGELELLI, J., TENO, J. M., & MILLER, S. C. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00309.x · Cited by 488
Self-citationAvidan, A. Y., Fries, B. E., James, M. L., Szafara, K. L., Wright, G. T., & Chervin, R. D. (2005) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2005.53304.x · Cited by 343
Klabunde, C. N., Vernon, S. W., Nadel, M. R., Breen, N., Seeff, L. C., & Brown, M. L. (2005) · Medical Care · doi:10.1097/01.mlr.0000173599.67470.ba · Cited by 287
The Residential History File: Studying Nursing Home Residents' Long‐Term Care Histories*
Intrator, O., Hiris, J., Berg, K., Miller, S. C., & Mor, V. (2011) · Health Services Research · doi:10.1111/j.1475-6773.2010.01194.x · Cited by 256
Self-citation
Metadata
- Authors
- Mor, V., Angelelli, J., Jones, R., Roy, J., Moore, T., & Morris, J.
- Venue
- BMC Health Services Research
- Published
- 2003-11-04
- Volume
- 3
- Issue
- 1
- Pages
- 20-20
- ISSN
- 1472-6963 (ISSN-L)
- OpenAlex
- W2155823288
- Cited by
- 197
- References
- 2
- FWCI
- 14.95
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Inter-rater reliability of nursing home quality indicators in the U.S. https://app.cochat.ai/papers/10.1186/1472-6963-3-20
- The paper
- Mor, V., Angelelli, J., Jones, R., Roy, J., Moore, T., & Morris, J. (2003). Inter-rater reliability of nursing home quality indicators in the U.S. BMC Health Services Research. https://doi.org/10.1186/1472-6963-3-20
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.