CoChat

How Robust Are Hospital Ranks Based on Composite Performance Measures?

Jacobs, Rowena, Goddard, Maria, Smith, Peter C. · 2005 · Medical Care
Cited by 107

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Composite indices of healthcare performance are an aggregation of underlying individual performance measures. They are increasingly being used to rank healthcare organizations. Although composite indicators present the "big picture" in a way that is easy to interpret, misleading conclusions may be drawn if attention is not paid to key methodological issues in their construction. OBJECTIVES: We examine variability in performance measures in the context of the construction and use of composite measures. We illustrate how variability in the underlying data and the resulting composite may undermine the robustness of performance measures in health care. We also illustrate how variation in the methodological rules applied to aggregate the individual indicators can have an important impact on composite scores. METHODS: We use data for 117 English acute hospitals to illustrate the generic methodological issues. The variance in performance measures is partitioned into "controllable" and "uncontrollable" elements. We create a composite index from the underlying performance indicators and use Monte Carlo simulations to examine the robustness of the composite. RESULTS: Random variation beyond the control of organizations gives rise to considerable uncertainty in hospital scores. Composites are also sensitive to changes made to the weighting system and to the aggregation rules. Some hospitals can jump almost half of the league table as a result of subtle changes. CONCLUSIONS: Great care is warranted in interpreting the results of composite performance measures. Suggestions for their future development are made.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1097/01.mlr.0000185692.72905.4a

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 2 references

Cited by

See all 107

Metadata

Authors
Jacobs, R., Goddard, M., & Smith, P. C.
Venue
Medical Care
Published
2005-12
Volume
43
Issue
12
Pages
1177-1184
ISSN
0025-7079 (ISSN-L)
OpenAlex
W1982670407
Cited by
107
References
2
FWCI
10.05
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: How Robust Are Hospital Ranks Based on Composite Performance Measures? https://app.cochat.ai/papers/10.1097/01.mlr.0000185692.72905.4a
The paper
Jacobs, R., Goddard, M., & Smith, P. C. (2005). How Robust Are Hospital Ranks Based on Composite Performance Measures? Medical Care. https://doi.org/10.1097/01.mlr.0000185692.72905.4a

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.