Quality and Outcomes Framework: what have we learnt?:
Open accessAbstract
Martin Roland and Bruce Guthrie assess the successes and failures of the pay-for-performance scheme and what its future should be In 2004 the UK National Health Service introduced the largest health related pay-for- performance scheme in the world—the Quality and Outcomes Framework (QOF).1 However Scotland is now abandoning the scheme, and growing disenchantment in England is likely to lead to major changes. What have we learnt, and what should happen to QOF in future? In the late 1990s, general practitioners’ pay had fallen substantially behind that of specialists, and morale and recruitment in general practice were poor. The government and the British Medical Association (BMA) privately agreed that a large pay rise was needed. Money was available because in 2000 the government had committed to increasing NHS spending to mid-European levels as a percentage of gross domestic product. However, the profession had to give something in return, and the BMA dropped its longstanding opposition to “quality payments” and started to negotiate a pay-for-performance scheme that would substantially increase funding for general practice. There followed 18 months of negotiations between BMA and NHS Employers with a small number of clinical advisers to develop the outcome measures (indicators) that would form the basis of the scheme (box 1). A starting premise was that the clinical indicators should be based on evidence based guidelines so that they would be likely to command a wide degree of professional support (box 2). The framework also included indicators related to practice organisation and patient participation. The package was controversial, and the BMA allowed its members to vote on the scheme—once in outline and once when the details were known. #### Box 1: How the Quality and Outcomes Framework works
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Pay-for-performance models are increasingly advocated in the US and internationally.
“Pay-for-performance (now more often termed”
From Measuring and improving the quality of mental health care: a global perspective (2018) · cites this paper as [89]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Correspondence - Tranexamic acid for traumatic brain injury
(2005) · London School of Economics and Political Science Theses Online (London School of Economics and Political Science) · doi:10.1016/s0140-6736 · Cited by 11,894 · see citing works
Effects of Pay for Performance on the Quality of Primary Care in England
Campbell, S., Reeves, D., Kontopantelis, E., Sibbald, B., & Roland, M. O. (2009) · New England Journal of Medicine · doi:10.1056/nejmsa0807651 · Cited by 695 · see citing works
Self-citationLinking physicians’ pay to the quality of care--a major experiment in the United Kingdom
Roland, M. (2004) · New England Journal of Medicine · doi:10.1056/nejmhpr041294 · Cited by 689 · see citing works
Self-citationDoran, T., Fullwood, C., Kontopantelis, E., & Reeves, D. (2008) · The Lancet · doi:10.1016/s0140-6736(08)61123-x · Cited by 263 · see citing works
Doran, T., Kontopantelis, E., Valderas, J. M., Campbell, S., Roland, M., Salisbury, C., & Reeves, D. (2011) · BMJ · doi:10.1136/bmj.d3590 · Cited by 258 · see citing works
Self-citationMcDonald, R., Harrison, S. A., Checkland, K., Campbell, S., & Roland, M. O. (2007) · BMJ · doi:10.1136/bmj.39238.890810.be · Cited by 195 · see citing works
Self-citationSafer Prescribing — A Trial of Education, Informatics, and Financial Incentives
Dreischulte, T., Donnan, P. T., Grant, A. M., Hapca, A., McCowan, C., & Guthrie, B. (2016) · New England Journal of Medicine · doi:10.1056/nejmsa1508955 · Cited by 195 · see citing works
Self-citationExclusion of Patients from Pay-for-Performance Targets by English Physicians
Doran, T., Fullwood, C., Reeves, D., Gravelle, H. S. E., & Roland, M. O. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa0800310 · Cited by 166 · see citing works
Self-citationCalvert, M., Shankar, A., McManus, R. J., Lester, H. E., & Freemantle, N. (2009) · BMJ · doi:10.1136/bmj.b1870 · Cited by 157 · see citing works
Lester, H. E., Schmittdiel, J. A., Selby, J. V., Fireman, B. H., Campbell, S., Lee, J., Whippy, A., & Madvig, P. (2010) · BMJ · doi:10.1136/bmj.c1898 · Cited by 153 · see citing works
Roland, M. O., Elliott, M. N., Lyratzopoulos, G., Barbiere, J. M., Parker, R. A., Smith, P., Bower, P., & Campbell, J. (2009) · BMJ · doi:10.1136/bmj.b3851 · Cited by 153 · see citing works
Self-citationRyan, A. M., Krinsky, S., Kontopantelis, E., & Doran, T. (2016) · The Lancet · doi:10.1016/s0140-6736(16)00276-2 · Cited by 151 · see citing works
Harrison, M. J., Dusheiko, M., Sutton, M., Gravelle, H. S. E., Doran, T., & Roland, M. O. (2014) · BMJ · doi:10.1136/bmj.g6423 · Cited by 140 · see citing works
Self-citationWorkload and reward in the Quality and Outcomes Framework of the 2004 general practice contract.
Guthrie, B., McLean, G., & Sutton, M. (2006) · Europe PMC (PubMed Central) · Cited by 71 · see citing works
Self-citationDoes pay-for-performance in primary care save lives?
Roland, M. O. (2016) · The Lancet · doi:10.1016/s0140-6736(16)00550-x · Cited by 11 · see citing works
Self-citationShould doctors be able to exclude patients from pay-for-performance schemes?
Roland, M. O. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-005003 · Cited by 10 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Type 2 diabetes and incidence of cardiovascular diseases: a cohort study in 1.9 million people
Shah, A. D., Langenberg, C., Rapsomaniki, E., Denaxas, S., Pujades-Rodriguez, M., Gale, C. P., Deanfield, J., Smeeth, L., Timmis, A., & Hemingway, H. (2015) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(14)70219-0 · Cited by 1,225 · see citing works
Measuring and improving the quality of mental health care: a global perspective
Kilbourne, A. M., Beck, K., Spaeth‐Rublee, B., Ramanuj, P., O'Brien, R. W., Tomoyasu, N., & Pincus, H. A. (2018) · World Psychiatry · doi:10.1002/wps.20482 · Cited by 457 · see citing works
Maximising the impact of patient reported outcome assessment for patients and society
Calvert, M., Kyte, D., Price, G., Valderas, J. M., & Hjollund, N. H. (2019) · BMJ · doi:10.1136/bmj.k5267 · Cited by 328 · see citing works
Are noise and air pollution related to the incidence of dementia? A cohort study in London, England
Carey, I. M., Anderson, H. R., Atkinson, R. W., Beevers, S. D., Cook, D. G., Strachan, D. P., Dajnak, D., Gulliver, J., & Kelly, F. J. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-022404 · Cited by 302 · see citing works
Forbes, L. J., Marchand, C., Doran, T., & Peckham, S. (2017) · British Journal of General Practice · doi:10.3399/bjgp17x693077 · Cited by 187 · see citing works
Palladino, R., Tabak, A. G., Khunti, K., Valabhji, J., Majeed, A., Millett, C., & Vamos, E. P. (2020) · BMJ Open Diabetes Research & Care · doi:10.1136/bmjdrc-2019-001061 · Cited by 98 · see citing works
Trends, Determinants, and Associations of Treated Hypothyroidism in the United Kingdom, 2005–2014
Razvi, S., Korevaar, T. I. M., & Taylor, P. (2019) · Thyroid · doi:10.1089/thy.2018.0251 · Cited by 45 · see citing works
Kypridemos, C., Bandosz, P., Hickey, G. L., Guzman-Castillo, M., Allen, K., Buchan, I., Capewell, S., & O’Flaherty, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0123112 · Cited by 15 · see citing works
Ramanuj, P. P., Breslau, J., Strathdee, G., Spaeth-Rublee, B., & Pincus, H. A. (2017) · Psychiatric Services · doi:10.1176/appi.ps.201600095 · Cited by 5 · see citing works
- Authors
- Roland, M., & Guthrie, B.
- Affiliations
- University of Cambridge (GB)
- University of Dundee (GB)
- Corresponding author
- Roland, Martin
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2016-08-04
- Volume
- 354
- Pages
- i4060
- ISSN
- 1756-1833 (ISSN-L)
- PMC
- PMC4975019
- OpenAlex
- W2491107951
- Cited by
- 215
- References
- 16
- Citations by year
- 2026: 10 · 2025: 8 · 2024: 29 · 2023: 15 · 2022: 11 · 2021: 23 · 2020: 27 · 2019: 31 · 2018: 25 · 2017: 11 · 2016: 8 · 2015: 13 · 2014: 4
- FWCI
- 46.57
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- BMJ (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Discovery Research Portal (University of Dundee) (publishedVersion, cc-by-nc)
- Apollo (University of Cambridge) (submittedVersion, cc-by-nc)
- HighWire Press Open Archive (submittedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by-nc)
- Apollo
- Topics
- Health Sciences › Health Professions › General Health Professions › Primary Care and Health Outcomes
- Health Sciences › Health Professions › General Health Professions › Health Services Management and Policy
- Health Sciences › Health Professions › General Health Professions › Healthcare Systems and Challenges
- Keywords
- Quality and Outcomes Framework, performance-based pay, general practice, clinical indicators, outcome measures, evidence-based recommendations, British Medical Association, NHS funding, patient participation, practice organization, Scotland, England
- MeSH terms
- United Kingdom, Humans, Quality of Health Care, Reimbursement, Incentive, State Medicine, Outcome Assessment, Health Care, Quality Indicators, Health Care, General Practice, Quality Improvement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
