How do aggregated patient-reported outcome measures data stimulate health care improvement? A realist synthesis
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Objectives: Internationally, there has been considerable debate about the role of data in supporting quality improvement in health care. Our objective was to understand how, why and in what circumstances the feedback of aggregated patient-reported outcome measures data improved patient care. Methods: We conducted a realist synthesis. We identified three main programme theories underlying the use of patient-reported outcome measures as a quality improvement strategy and expressed them as nine ‘if then’ propositions. We identified international evidence to test these propositions through searches of electronic databases and citation tracking, and supplemented our synthesis with evidence from similar forms of performance data. We synthesized this evidence through comparing the mechanisms and impact of patient-reported outcome measures and other performance data on quality improvement in different contexts. Results: Three programme theories were identified: supporting patient choice, improving accountability and enabling providers to compare their performance with others. Relevant contextual factors were extent of public disclosure, use of financial incentives, perceived credibility of the data and the practicality of the results. Available evidence suggests that patients or their agents rarely use any published performance data when selecting a provider. The perceived motivation behind public reporting is an important determinant of how providers respond. When clinicians perceived that performance indicators were not credible but were incentivized to collect them, gaming or manipulation of data occurred. Outcome data do not provide information on the cause of poor care: providers needed to integrate and interpret patient-reported outcome measures and other outcome data in the context of other data. Lack of timeliness of performance data constrains their impact. Conclusions: Although there is only limited research evidence to support some widely held theories of how aggregated patient-reported outcome measures data stimulate quality improvement, several lessons emerge from interventions sharing the same programme theories to help guide the increasing use of these measures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Foucault, M. (1982) · Critical Inquiry · doi:10.1086/448181 · Cited by 8,496
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882
Foucault, M. (1998) · Ethische und politische Freiheit · doi:10.1515/9783110815764.387 · Cited by 3,248
Pawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791
RAMESES publication standards: realist syntheses
Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-21 · Cited by 1,374
Fung, C. H., Lim, Y.-W., Mattke, S., Damberg, C., & Shekelle, P. G. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-2-200801150-00006 · Cited by 980 · cited 2×
“Previous reviews of quantitative evidence suggest there is little support for the ‘selection’ pathway but there is evidence to support the ‘change’ pathway.(11,12) These reviews show that changes made in response to performance data vary in their impact on provider behaviour and health outcomes, while other work has n…”
KAPLAN, H. C., BRADY, P. W., DRITZ, M. C., HOOPER, D. K., LINAM, W. M., FROEHLE, C. M., & MARGOLIS, P. (2010) · Milbank Quarterly · doi:10.1111/j.1468-0009.2010.00611.x · Cited by 714 · cited 1×
“Although testing these theories led us to consider how some characteristics of organizations supported or limited the success of the implementation of quality improvement activities, we did not comprehensively review this literature.(52) Similarly, there are other explanations for why patients may not utilize quality…”
BackgroundConnections Between Quality Measurement and Improvement
Berwick, D. M., James, B., & Coye, M. J. (2003) · Medical Care · doi:10.1097/00005650-200301001-00004 · Cited by 575 · cited 1×
“It has been hypothesized that such data can improve the quality and outcomes of patient care either through a ‘selection’ pathway (whereby patients choose a high-quality hospital) or through a ‘change’ pathway (whereby providers initiate quality improvement).(8) Provider initiatives include those seeking to preserve t…”
BackgroundHospital Performance Reports: Impact On Quality, Market Share, And Reputation
Hibbard, J. H., Stockard, J., & Tusler, M. (2005) · Health Affairs · doi:10.1377/hlthaff.24.4.1150 · Cited by 413 · cited 2×
“There is evidence to indicate that public reporting places additional pressure on providers to respond, particularly poor performers (Proposition 2).(9,32) When data were fed back privately, with no public reporting or financial incentives, providers either ignored them or focused on altering their data collection pra…”
BackgroundCollecting data on patient experience is not enough: they must be used to improve care
Coulter, A., Locock, L., Ziebland, S., & Calabrese, J. (2014) · BMJ · doi:10.1136/bmj.g2225 · Cited by 409 · cited 1×
“We also acknowledge that patient experience data enable providers to understand what matters to patients more generally which may inform training, rather than, or in addition to, quality improvement activities.(54) Literature on the use of aggregated PROMs data is limited as the use of these measures for quality assur…”
Background
Metadata
- Authors
- Greenhalgh, J., Dalkin, S., Gibbons, E., Wright, J., Valderas, J. M., Meads, D., & Black, N.
- Venue
- Journal of Health Services Research & Policy
- Publisher
- SAGE Publications
- Published
- 2017-12-20
- Volume
- 23
- Issue
- 1
- Pages
- 57-65
- ISSN
- 1355-8196 (ISSN-L)
- 1758-1060
- OpenAlex
- W2779490288
- Cited by
- 178
- References
- 41
- FWCI
- 40.10
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How do aggregated patient-reported outcome measures data stimulate health care improvement? A realist synthesis. https://app.cochat.ai/papers/10.1177/1355819617740925
- The paper
- Greenhalgh, J., Dalkin, S., Gibbons, E., Wright, J., Valderas, J. M., Meads, D., & Black, N. (2018). How do aggregated patient-reported outcome measures data stimulate health care improvement? A realist synthesis. Journal of Health Services Research & Policy. https://doi.org/10.1177/1355819617740925
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.