The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Functionality and feedback: a realist synthesis of the collation, interpretation and utilisation of patient-reported outcome measures data to improve patient care
Open accessAbstract
Background: The feedback of patient-reported outcome measures (PROMs) data is intended to support the care of individual patients and to act as a quality improvement (QI) strategy. Objectives: To (1) identify the ideas and assumptions underlying how individual and aggregated PROMs data are intended to improve patient care, and (2) review the evidence to examine the circumstances in which and processes through which PROMs feedback improves patient care. Design: Two separate but related realist syntheses: (1) feedback of aggregate PROMs and performance data to improve patient care, and (2) feedback of individual PROMs data to improve patient care. Interventions: Aggregate – feedback and public reporting of PROMs, patient experience data and performance data to hospital providers and primary care organisations. Individual – feedback of PROMs in oncology, palliative care and the care of people with mental health problems in primary and secondary care settings. Main outcome measures: Aggregate – providers’ responses, attitudes and experiences of using PROMs and performance data to improve patient care. Individual – providers’ and patients’ experiences of using PROMs data to raise issues with clinicians, change clinicians’ communication practices, change patient management and improve patient well-being. Data sources: Searches of electronic databases and forwards and backwards citation tracking. Review methods: Realist synthesis to identify, test and refine programme theories about when, how and why PROMs feedback leads to improvements in patient care. Results: Providers were more likely to take steps to improve patient care in response to the feedback and public reporting of aggregate PROMs and performance data if they perceived that these data were credible, were aimed at improving patient care, and were timely and provided a clear indication of the source of the problem. However, implementing substantial and sustainable improvement to patient care required system-wide approaches. In the care of individual patients, PROMs function more as a tool to support patients in raising issues with clinicians than they do in substantially changing clinicians’ communication practices with patients. Patients valued both standardised and individualised PROMs as a tool to raise issues, but thought is required as to which patients may benefit and which may not. In settings such as palliative care and psychotherapy, clinicians viewed individualised PROMs as useful to build rapport and support the therapeutic process. PROMs feedback did not substantially shift clinicians’ communication practices or focus discussion on psychosocial issues; this required a shift in clinicians’ perceptions of their remit. Strengths and limitations: There was a paucity of research examining the feedback of aggregate PROMs data to providers, and we drew on evidence from interventions with similar programme theories (other forms of performance data) to test our theories. Conclusions: PROMs data act as ‘tin openers’ rather than ‘dials’. Providers need more support and guidance on how to collect their own internal data, how to rule out alternative explanations for their outlier status and how to explore the possible causes of their outlier status. There is also tension between PROMs as a QI strategy versus their use in the care of individual patients; PROMs that clinicians find useful in assessing patients, such as individualised measures, are not useful as indicators of service quality. Future work: Future research should (1) explore how differently performing providers have responded to aggregate PROMs feedback, and how organisations have collected PROMs data both for individual patient care and to improve service quality; and (2) explore whether or not and how incorporating PROMs into patients’ electronic records allows multiple different clinicians to receive PROMs feedback, discuss it with patients and act on the data to improve patient care. Study registration: This study is registered as PROSPERO CRD42013005938. Funding: The National Institute for Health Research Health Services and Delivery Research programme.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Aaronson, N. K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., Duez, N. J., Filiberti, A., Flechtner, H., Fleishman, S. B., Haes, J. C. J. M. D., et al. (1993) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/85.5.365 · Cited by 16,005
Strategic Responses to Institutional Processes
Oliver, C. (1991) · The Academy of Management Review · doi:10.5465/amr.1991.4279002 · Cited by 8,950
Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations
GREENHALGH, T., ROBERT, G., MACFARLANE, F., BATE, P., & KYRIAKIDOU, O. (2004) · The Milbank Quarterly · doi:10.1111/j.0887-378x.2004.00325.x · Cited by 7,448 · cited 1×
“The exploration of key factors involved in this process was beyond the scope of this review, but it has been explored in other reviews.(89,366,367)”
BackgroundMultitask principal – agent analyses: Incentive contracts, asset ownership, and job design
Holmstrom, B., & Milgrom, P. (1991) · The Journal of Law, Economics, and Organization · doi:10.1093/jleo/7.special_issue.24 · Cited by 6,046
The functional assessment of cancer therapy scale: development and validation of the general measure
Cella, D. F., Tulsky, D. S., Gray, G., Sarafian, B., Linn, E., Bonomi, A., Silberman, M., Yellen, S. B., Winicour, P., & Brannon, J. (1993) · Journal of Clinical Oncology · doi:10.1200/jco.1993.11.3.570 · Cited by 6,020
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882
Audit and feedback: effects on professional practice and healthcare outcomes
Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard-Jensen, J., French, S. D., O'Brien, M. A., Johansen, M., Grimshaw, J., & Oxman, A. D. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000259.pub3 · Cited by 5,485
Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias
Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1991) · Journal of Economic Perspectives · doi:10.1257/jep.5.1.193 · Cited by 5,019
Jaeschke, R., Singer, J., & Guyatt, G. H. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90005-6 · Cited by 4,982
Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., Bouter, L. M., & de Vet, H. C. W. (2010) · Quality of Life Research · doi:10.1007/s11136-010-9606-8 · Cited by 4,260
Wilson, I. B. (1995) · JAMA · doi:10.1001/jama.1995.03520250075037 · Cited by 3,027 · cited 1×
“Attempts to address this conundrum included the development of models to show the pathway through which changes to clinical variables impacted on symptoms, which in turn impacted on a patient’s functional status, general health perceptions and, ultimately, overall quality of life.(250)”
MethodCarver, C. S., & Scheier, M. F. (1982) · Psychological Bulletin · doi:10.1037//0033-2909.92.1.111 · Cited by 2,650
Doctor-patient communication: a review of the literature
Ong, L. M. L., de Haes, J. C. J. M., Hoos, A. M., & Lammes, F. B. (1995) · Social Science & Medicine · doi:10.1016/0277-9536(94)00155-m · Cited by 2,430 · cited 4×
“As Ong et al.(302) argued, the clinician–patient consultation serves a number of functions, which include (1) creating a good interpersonal relationship, (2) exchanging information and (3) making treatment-related decisions.”
BackgroundInfluentialIndividual-patient monitoring in clinical practice: are available health status surveys adequate?
McHorney, C. A., & Tarlov, A. R. (1995) · Quality of Life Research · doi:10.1007/bf01593882 · Cited by 2,206
Patient reported outcome measures could help transform healthcare
Black, N. (2013) · BMJ · doi:10.1136/bmj.f167 · Cited by 1,960
Self-citationPawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791
Self-citationWHAT’S MEASURED IS WHAT MATTERS: TARGETS AND GAMING IN THE ENGLISH PUBLIC HEALTH CARE SYSTEM
BEVAN, G., & HOOD, C. (2006) · Public Administration · doi:10.1111/j.1467-9299.2006.00600.x · Cited by 1,580
Velikova, G., Booth, L., Smith, A. B., Brown, P. M., Lynch, P., Brown, J. M., & Selby, P. J. (2004) · Journal of Clinical Oncology · doi:10.1200/jco.2004.06.078 · Cited by 1,382
RAMESES publication standards: realist syntheses
Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-21 · Cited by 1,374
Self-citationPatrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E., & Ring, L. (2011) · Value in Health · doi:10.1016/j.jval.2011.06.014 · Cited by 1,095
Detmar, S. B., Muller, M. J., Schornagel, J. H., Wever, L. D. V., & Aaronson, N. K. (2002) · JAMA · doi:10.1001/jama.288.23.3027 · Cited by 1,011
Audit and feedback: effects on professional practice and health care outcomes
Jamtvedt, G., Young, J. M., Kristoffersen, D. T., O'Brien, M. A., & Oxman, A. D. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000259.pub2 · Cited by 912 · cited 2×
“Audit and feedback Patient-reported outcome measures feedback is an example of an audit and feedback intervention, as it involves generating a ‘summary of clinical performance of health care over a specified period of time aimed at providing information to health professionals to allow them to assess and adjust their…”
BackgroundMethodWhen words are painful: unraveling the mechanisms of the nocebo effect
Benedetti, F., Lanotte, M., Lopiano, L., & Colloca, L. (2007) · Neuroscience · doi:10.1016/j.neuroscience.2007.02.020 · Cited by 636 · cited 1×
“also drew attention to the possibility of the ‘nocebo effect’, defined as ‘the opposite to the placebo effect, whereby expectation of a negative outcome may lead to the worsening of a symptom’.(298) Thombs et al.”
BackgroundIs it time for clinicians to routinely track patient outcome? A meta-analysis
Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.bpg025 · Cited by 608
Impact of patient‐reported outcome measures on routine practice: a structured review
Marshall, S., Haywood, K., & Fitzpatrick, R. (2006) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2006.00650.x · Cited by 602 · cited 2×
“Systematic reviews of PROMs feedback in the care of individual patients with long-term health conditions suggest that PROMs feedback leads to increased discussion of problems during the consultation, and helps clinicians to identify and detect patient problems, but has less impact on patient management or patient outc…”
BackgroundSteventon, A., Bardsley, M., Billings, J., Dixon, J., Doll, H., Hirani, S., Cartwright, M., Rixon, L., Knapp, M., Henderson, C., et al. (2012) · BMJ · doi:10.1136/bmj.e3874 · Cited by 583
A New Approach to the Measurement of Quality of Life: The Patient-Generated Index
Ruta, D. A., Garratt, A. M., Leng, M., Russell, I. T., & MacDonald, L. M. (1994) · Medical Care · doi:10.1097/00005650-199411000-00004 · Cited by 527
Flodgren, G., Eccles, M. P., Shepperd, S., Scott, A., Parmelli, E., & Beyer, F. R. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009255 · Cited by 418 · cited 1×
“A number of systematic reviews have highlighted the variable impact of financial incentives on professional behaviour.(213,214) Pawson’s programme theory of public disclosure programmes also recognises that publicising performance data rarely works in isolation from other sanctions.”
BackgroundMeasuring patient-reported outcomes: moving beyond misplaced common sense to hard science
McKenna, S. P. (2011) · BMC Medicine · doi:10.1186/1741-7015-9-86 · Cited by 319 · cited 1×
“Instrument developers usually advise against the rewording or omission of items from standardised PROMs, as this may threaten their psychometric properties.(345)”
BackgroundDoran, T., Kontopantelis, E., Valderas, J. M., Campbell, S., Roland, M., Salisbury, C., & Reeves, D. (2011) · BMJ · doi:10.1136/bmj.d3590 · Cited by 258
Self-citationCartwright, M., Hirani, S. P., Rixon, L., Beynon, M., Doll, H., Bower, P., Bardsley, M., Steventon, A., Knapp, M., Henderson, C., et al. (2013) · BMJ · doi:10.1136/bmj.f653 · Cited by 240
Gardner, B., Whittington, C., McAteer, J., Eccles, M. P., & Michie, S. (2010) · Social Science & Medicine · doi:10.1016/j.socscimed.2010.01.039 · Cited by 225
New Labour, modernisation and the medical labour process
HARRISON, S. (2002) · Journal of Social Policy · doi:10.1017/s0047279402006694 · Cited by 214 · cited 1×
“At the individual level, PROMs feedback for detection and monitoring of patients’ problems can be seen as an attempt to modify clinical judgement with encoded, standardised knowledge as part of the move towards scientific-bureaucratic medicine.(22) The intention of increasing patient involvement in the consultation be…”
BackgroundPatient and Public Involvement in Patient-Reported Outcome Measures: Evolution Not Revolution
Staniszewska, S., Haywood, K. L., Brett, J., & Tutton, L. (2012) · The Patient: Patient-Centered Outcomes Research · doi:10.2165/11597150-000000000-00000 · Cited by 196
van der Veer, S. N., de Keizer, N. F., Ravelli, A. C. J., Tenkink, S., & Jager, K. J. (2010) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2010.01.011 · Cited by 184
Neudert, C., Wasner, M., & Borasio, G. D. (2001) · Journal of the Neurological Sciences · doi:10.1016/s0022-510x(01)00612-8 · Cited by 157 · cited 4×
“Neudert et al.(329) This study assessed patients’ perceptions of the validity and distress caused by completion of three different PROMs: the Sickness Impact Profile (SIP), the SF-36(252) and the Schedule for the Evaluation of Individual Quality of Life Direct Weight (SEIQoL-DW).”
BackgroundResultInfluentialBørøsund, E., Cvancarova, M., Moore, S. M., Ekstedt, M., & Ruland, C. M. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3348 · Cited by 148
Patient-reported outcome measures: Are they patient-generated, patient-centred or patient-valued?
Trujols, J., Portella, M. J., Iraurgi, I., Campins, M. J., Siñol, N., & Cobos, J. P. D. L. (2013) · Journal of Mental Health · doi:10.3109/09638237.2012.734653 · Cited by 129
Cited by
Greenhalgh, J., Gooding, K., Gibbons, E., Dalkin, S., Wright, J., Valderas, J., & Black, N. (2018) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-018-0061-6 · Cited by 588
Self-citationDi Maio, M., Basch, E., Denis, F., Fallowfield, L. J., Ganz, P. A., Howell, D., Kowalski, C., Perrone, F., Stover, A. M., Sundaresan, P., et al. (2022) · Annals of Oncology · doi:10.1016/j.annonc.2022.04.007 · Cited by 509
Brown, B., Gude, W. T., Blakeman, T., van der Veer, S. N., Ivers, N., Francis, J. J., Lorencatto, F., Presseau, J., Peek, N., & Daker-White, G. (2019) · Implementation Science · doi:10.1186/s13012-019-0883-5 · Cited by 387
Foster, A., Croot, L., Brazier, J., Harris, J., & O’Cathain, A. (2018) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-018-0072-3 · Cited by 386
Maximising the impact of patient reported outcome assessment for patients and society
Calvert, M., Kyte, D., Price, G., Valderas, J. M., & Hjollund, N. H. (2019) · BMJ · doi:10.1136/bmj.k5267 · Cited by 328
Self-citation
Metadata
- Authors
- Greenhalgh, J., Dalkin, S., Gooding, K., Gibbons, E., Wright, J., Meads, D., Black, N., Valderas, J. M., & Pawson, R.
- Venue
- Health Services and Delivery Research
- Published
- 2017-01
- Volume
- 5
- Issue
- 2
- Pages
- 1-280
- ISSN
- 2050-4349 (ISSN-L)
- 2050-4357
- OpenAlex
- W2581606077
- Cited by
- 234
- References
- 38
- FWCI
- 38.04
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.