The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
Research methods used in developing and applying quality indicators in primary care
Open accessAbstract
Quality indicators have been developed throughout Europe primarily for use in hospitals, but also increasingly for primary care. Both development and application are important but there has been less research on the application of indicators. Three issues are important when developing or applying indicators: (1). which stakeholder perspective(s) are the indicators intended to reflect; (2). what aspects of health care are being measured; and (3). what evidence is available? The information required to develop quality indicators can be derived using systematic or non-systematic methods. Non-systematic methods such as case studies play an important role but they do not tap in to available evidence. Systematic methods can be based directly on scientific evidence by combining available evidence with expert opinion, or they can be based on clinical guidelines. While it may never be possible to produce an error free measure of quality, measures should adhere, as far as possible, to some fundamental a priori characteristics (acceptability, feasibility, reliability, sensitivity to change, and validity). Adherence to these characteristics will help maximise the effectiveness of quality indicators in quality improvement strategies. It is also necessary to consider what the results of applying indicators tell us about quality of care.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Health Measurement Scales: A Practical Guide to Their Development and Use
Streiner, D. L., Norman, G. R., & Cairney, J. (1989) · Cited by 9,595
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951
The National Institute for Clinical Excellence
(2001) · Nursing Management · doi:10.7748/nm.8.2.39.s15 · Cited by 3,876
Qualitative Research: Consensus methods for medical and health services research
Jones, J., & Hunter, D. (1995) · BMJ · doi:10.1136/bmj.311.7001.376 · Cited by 3,471
Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations
Grimshaw, J. M., & Russell, I. T. (1993) · The Lancet · doi:10.1016/0140-6736(93)92244-n · Cited by 2,770
Consensus methods: characteristics and guidelines for use
Fink, A., Kosecoff, J., Chassin, M., & Brook, R. H. (1984) · American Journal of Public Health · doi:10.2105/ajph.74.9.979 · Cited by 2,089
Consensus development methods, and their use in clinical guideline development.
Murphy, Black, Lamping, McKee, Sanderson, Askham, & Marteau (1998) · Health Technology Assessment · doi:10.3310/hta2030 · Cited by 2,051
American Academy of Family Physicians
(1966) · New England Journal of Medicine · doi:10.1056/nejm196611172752014 · Cited by 1,948
Health Measurement Scales--A Practical Guide to Their Development and Use
Waclawski, E. R. (2010) · Occupational Medicine · doi:10.1093/occmed/kqp179 · Cited by 1,874
The National Committee for Quality Assurance
Iglehart, J. K. (1996) · New England Journal of Medicine · doi:10.1056/nejm199609263351322 · Cited by 1,824
THE ROYAL AUSTRALIAN COLLEGE OF GENERAL PRACTITIONERS
(1974) · Medical Journal of Australia · doi:10.5694/j.1326-5377.1974.tb93279.x · Cited by 1,341
Ferlie, E. B., & Shortell, S. M. (2001) · The Milbank Quarterly · doi:10.1111/1468-0009.00206 · Cited by 1,335
Juran, J. M. (1988) · Cited by 1,209
Campbell, S. M., Roland, M. O., & Buetow, S. A. (2000) · Social Science & Medicine · doi:10.1016/s0277-9536(00)00057-5 · Cited by 1,170
Self-citationThe Medical Outcomes Study: An Application of Methods for Monitoring the Results of Medical Care
Tarlov, A. R. (1989) · JAMA · doi:10.1001/jama.1989.03430070073033 · Cited by 877
Grol, R., Dalhuijsen, J., Thomas, S., Veld, C. I. ‘., Rutten, G., & Mokkink, H. (1998) · BMJ · doi:10.1136/bmj.317.7162.858 · Cited by 877
National service framework for coronary heart disease
Lloyd-Mostyn, R. (2000) · BMJ · doi:10.1136/bmj.321.7261.634 · Cited by 832
Brook, R. H., McGlynn, E. A., & Cleary, P. D. (1996) · New England Journal of Medicine · doi:10.1056/nejm199609263351311 · Cited by 769
A Method for the Detailed Assessment of the Appropriateness of Medical Technologies
Brook, R. H., Chassin, M. R., Fink, A., Solomon, D. H., Kosecoff, J., & Park, R. E. (1986) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462300002774 · Cited by 762
Grey zones of clinical practice: some limits to evidence-based medicine
Naylor, C. D. (1995) · The Lancet · doi:10.1016/s0140-6736(95)92969-x · Cited by 663
Donabedian, A. (1978) · Science · doi:10.1126/science.417400 · Cited by 571
Consensus Development Methods: A Review of Best Practice in Creating Clinical Guidelines
Black, N., Murphy, M., Lamping, D., McKee, M., Sanderson, C., Askham, J., & Marteau, T. (1999) · Journal of Health Services Research & Policy · doi:10.1177/135581969900400410 · Cited by 451
Identifying predictors of high quality care in English general practice: observational study
Campbell, S. M., Hann, M., Hacker, J., Burns, C., Oliver, D., Thapar, A., Mead, N., Safran, D. G., & Roland, M. O. (2001) · BMJ · doi:10.1136/bmj.323.7316.784 · Cited by 419
Self-citationWensing, M., Jung, H. P., Mainz, J., Olesen, F., & Grol, R. (1998) · Social Science & Medicine · doi:10.1016/s0277-9536(98)00222-6 · Cited by 369
The Reproducibility of a Method to Identify the Overuse and Underuse of Medical Procedures
Shekelle, P. G., Kahan, J. P., Bernstein, S. J., Leape, L. L., Kamberg, C. J., & Park, R. E. (1998) · New England Journal of Medicine · doi:10.1056/nejm199806253382607 · Cited by 368
Treatment of early Parkinson's disease
Marsh, L., & Dawson, T. M. (2000) · BMJ · doi:10.1136/bmj.321.7252.1 · Cited by 337
Luck, J., Peabody, J. W., Dresselhaus, T. R., Lee, M., & Glassman, P. (2000) · The American Journal of Medicine · doi:10.1016/s0002-9343(00)00363-6 · Cited by 335
Defining and measuring quality of care: a perspective from US researchers
BROOK, R. H., McGLYNN, E. A., & SHEKELLE, P. G. (2000) · International Journal for Quality in Health Care · doi:10.1093/intqhc/12.4.281 · Cited by 327
Mant, J., & Hicks, N. (1995) · BMJ · doi:10.1136/bmj.311.7008.793 · Cited by 303
Kahn, K. L. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03450150069033 · Cited by 293
Hemingway, H., Crook, A. M., Feder, G., Banerjee, S., Dawson, J. R., Magee, P., Philpott, S., Sanders, J., Wood, A., & Timmis, A. D. (2001) · New England Journal of Medicine · doi:10.1056/nejm200103013440906 · Cited by 262
Prescribing indicators for UK general practice: Delphi consultation study
Campbell, S. M. (2000) · BMJ · doi:10.1136/bmj.321.7258.425 · Cited by 249
Self-citationConsensus methods in prescribing research
Campbell, S. M., & Cantrill, J. A. (2001) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1046/j.1365-2710.2001.00331.x · Cited by 239
Self-citationAssessing Care of Vulnerable Elders: Methods for Developing Quality Indicators
Shekelle, P. G., MacLean, C. H., Morton, S. C., & Wenger, N. S. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-8_part_2-200110161-00003 · Cited by 232
Indicators of Quality in Health Care
Lawrence, M., & Olesen, F. (1997) · European Journal of General Practice · doi:10.3109/13814789709160336 · Cited by 224
Six Challenges In Measuring The Quality Of Health Care
McGlynn, E. A. (1997) · Health Affairs · doi:10.1377/hlthaff.16.3.7 · Cited by 223
North of England evidence based guidelines development project: methods of guideline development
Eccles, M., Clapp, Z., Grimshaw, J., Adams, P. C., Higgins, B., Purves, I., & Russell, I. (1996) · BMJ · doi:10.1136/bmj.312.7033.760 · Cited by 216
Seddon, M. E. (2001) · Quality in Health Care · doi:10.1136/qhc.0100152 · Cited by 215
The Methodologic Foundations of Studies of the Appropriateness of Medical Care
Phelps, C. E. (1993) · New England Journal of Medicine · doi:10.1056/nejm199310213291707 · Cited by 203
Selby, J. V., Fireman, B. H., Lundstrom, R. J., Swain, B. E., Truman, A. F., Wong, C. C., Froelicher, E. S., Barron, H. V., & Hlatky, M. A. (1996) · New England Journal of Medicine · doi:10.1056/nejm199612193352506 · Cited by 181
The General Practice Assessment Survey (GPAS): tests of data quality and measurement properties
Ramsay, J. (2000) · Family Practice · doi:10.1093/fampra/17.5.372 · Cited by 167
Validity of Criteria Used for Detecting Underuse of Coronary Revascularization
Kravitz, R. L. (1995) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1995.03530080048040 · Cited by 165
Ayanian, J. Z., Landrum, M. B., Normand, S.-L. T., Guadagnoli, E., & McNeil, B. J. (1998) · New England Journal of Medicine · doi:10.1056/nejm199806253382608 · Cited by 159
Campbell, S., Hann, M., Roland, M. O., Quayle, J. A., & Shekelle, P. (1999) · Medical Care · doi:10.1097/00005650-199909000-00012 · Cited by 157
Self-citationNormand, S. T., McNeil, B. J., Peterson, L. E., & Rh, P. (1998) · International Journal for Quality in Health Care · doi:10.1093/intqhc/10.3.247 · Cited by 144
Schers, H., Braspenning, J. C. C., Drijver, R., Wensing, M., & Grol, R. P. T. M. (2000) · Europe PMC (PubMed Central) · Cited by 143
Cantrill, J. A., Sibbald, B., & Buetow, S. (1998) · BMJ Quality & Safety · doi:10.1136/qshc.7.3.130 · Cited by 141
Performance indicators for primary care groups: an evidence based approach
McColl, A., Roderick, P., Gabbay, J., Smith, H. E., & Moore, M. V. (1998) · BMJ · doi:10.1136/bmj.317.7169.1354 · Cited by 135
Coulter, I. D., Adams, A. H., & Shekelle, P. (1997) · RAND Corporation eBooks · doi:10.7249/rp572 · Cited by 134
Some observations on attempts to measure appropriateness of care
Hicks, N. (1994) · BMJ · doi:10.1136/bmj.309.6956.730 · Cited by 129
Measuring quality in general practice. Pilot study of a needs, process and outcome measure.
Howie, J. G., Heaney, D., & Maxwell, M. (1997) · PubMed · Cited by 125
Derivation of clinical indications for carotid endarterectomy by an expert panel.
Merrick, N. J., Fink, A., Park, R. E., Brook, R. H., Kosecoff, J., Chassin, M. R., & Solomon, D. H. (1987) · American Journal of Public Health · doi:10.2105/ajph.77.2.187 · Cited by 119
Palmer, R. H. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-8_part_2-199710151-00059 · Cited by 111
Audit of coronary angiography and bypass surgery
Gray, D. P., Hampton, J. R., Bernstein, S. J., Brook, R. H., & Kosecoff, J. (1990) · The Lancet · doi:10.1016/0140-6736(90)91196-h · Cited by 110
Davies, H. T. O., & Crombie, I. K. (1995) · BMJ · doi:10.1136/bmj.311.7008.766 · Cited by 101
The role of evidence in the consensus process. Results from a Canadian consensus exercise
Lomas, J. (1988) · JAMA · doi:10.1001/jama.259.20.3001 · Cited by 90
Advancing Quality: Total Quality Management in the National Health Service
Joss, R., & Kogan, M. (1995) · Medical Entomology and Zoology · Cited by 85
Health Service Quality: An introduction to quality methods for health services
Hunter, A. (1992) · Physiotherapy · doi:10.1016/s0031-9406(10)61508-8 · Cited by 85
Design should help use of patients' data
Wyatt, J. C., & Wright, P. (1998) · The Lancet · doi:10.1016/s0140-6736(97)08306-2 · Cited by 82
Campbell, S., Roland, M. O., Shekelle, P., Cantrill, J. A., Buetow, S., & Cragg, D. K. (1999) · BMJ Quality & Safety · doi:10.1136/qshc.8.1.6 · Cited by 80
Self-citationCampbell, S., Roland, M. O., Quayle, J. A., Buetow, S., & Shekelle, P. (1998) · Journal of Public Health · doi:10.1093/oxfordjournals.pubmed.a024796 · Cited by 79
Self-citationWu, L., & Ashton, C. M. (1997) · Evaluation & the Health Professions · doi:10.1177/016327879702000203 · Cited by 77
Appropriateness of cholecystectomy in the United Kingdom--a consensus panel approach.
Scott, E. A., & Black, N. (1991) · Gut · doi:10.1136/gut.32.9.1066 · Cited by 72
Campbell, S. (2002) · BMJ Quality & Safety · doi:10.1136/qhc.11.2.125 · Cited by 69
Self-citationThe accuracy of medical record documentation in schizophrenia
Cradock, J. A., Young, A. S., & Sullivan, G. (2001) · The Journal of Behavioral Health Services & Research · doi:10.1007/bf02287775 · Cited by 63
Salzer, M. S., Nixon, C. T., Schut, L. J. A., Karver, M. S., & Bickman, L. (1997) · Evaluation Review · doi:10.1177/0193841x9702100302 · Cited by 62
Frijling, B. D., Spies, T. H., Lobo, C. M., Hulscher, M., Drenth, B. B. V., Braspenning, J. C. C., Prins, A., van der Wouden, J. C., & Grol, R. P. T. M. (2001) · Europe PMC (PubMed Central) · Cited by 61
Comprehensive Reliability Assessment and Comparison of Quality Indicators and Their Components
Huff, E. D. (1997) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00218-7 · Cited by 57
Smeele, I. J. M., Grol, R. P. T. M., van Schayck, O. C. P., van den Bosch, W. J., van den Hoogen, H. J., & Muris, J. W. M. (1999) · BMJ Quality & Safety · doi:10.1136/qshc.8.2.92 · Cited by 57
Reactions to the use of evidence-based performance indicators in primary care: a qualitative study
Wilkinson, E. K., McColl, A., Exworthy, M., Roderick, P., Smith, H., Moore, M. V., & Gabbay, J. (2000) · BMJ Quality & Safety · doi:10.1136/qhc.9.3.166 · Cited by 46
van den Hombergh, P., Grol, R., van den Hoogen, H. J., & van den Bosch, W. J. (1999) · Quality in Health Care · doi:10.1136/qshc.8.3.161 · Cited by 45
The Appropriateness of Coronary Artery Bypass Graft Surgery in Academic Medical Centers
Leape, L. L., Hilborne, L. H., Schwartz, J. S., Bates, D. W., Rubin, H. R., Slavin, P., Park, R. E., Witter, D. M., Jr., Panzer, R. J., & Brook, R. H. (1996) · Annals of Internal Medicine · doi:10.7326/0003-4819-125-1-199607010-00003 · Cited by 41
The outcomes utility index: will outcomes data tell us what we want to know?
McGlynn, E. A. (1998) · International Journal for Quality in Health Care · doi:10.1093/intqhc/10.6.485 · Cited by 37
Clinical governance in primary care: Participating in clinical governance
Pringle, M. (2000) · BMJ · doi:10.1136/bmj.321.7263.737 · Cited by 36
Managing for Quality in General Practice
Irvine, D. (1990) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 31
Pringle, M., & Jones, J. H. (1998) · BMJ · doi:10.1136/bmj.317.7166.1120 · Cited by 28
The role of clinical governance as a strategy for quality improvement in primary care.
Campbell, S., & Sweeney, G. (2002) · Europe PMC (PubMed Central) · Cited by 26
Self-citationModernising the NHS: A promising start, but fundamental reform is needed
Enthoven, A. C. (2000) · BMJ · doi:10.1136/bmj.320.7245.1329 · Cited by 22
Quality assessment for general practice Supporting clinical governance in primary care groups
Roland, M., Holden, J., Campbell, S., & Manchester Univ. (United Kingdom). National Primary Care Research and Development Centre (1998) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 4
Self-citation
Cited by
Lin, I., Wiles, L., Waller, R., Goucke, R., Nagree, Y., Gibberd, M., Straker, L., Maher, C. G., & O’Sullivan, P. P. B. (2020) · British Journal of Sports Medicine · doi:10.1136/bjsports-2018-099878 · Cited by 1,001
The Impact of Primary Care: A Focused Review
Shi, L. (2012) · Scientifica · doi:10.6064/2012/432892 · Cited by 535
Waggoner, J., Carline, J. D., & Durning, S. J. (2016) · Academic Medicine · doi:10.1097/acm.0000000000001092 · Cited by 429
Performance Measurement for Health System Improvement: Experiences, Challenges and Prospects
(2010) · Cambridge University Press eBooks · doi:10.1017/cbo9780511711800 · Cited by 426
Eubank, B. H., Mohtadi, N. G., Lafave, M. R., Wiley, J. P., Bois, A. J., Boorman, R. S., & Sheps, D. M. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0165-8 · Cited by 423
Metadata
- Authors
- Campbell, S. M.
- Venue
- Quality and Safety in Health Care
- Publisher
- BMJ
- Published
- 2002-12-01
- Volume
- 11
- Issue
- 4
- Pages
- 358-364
- ISSN
- 1475-3898 (ISSN-L)
- 1475-3901
- OpenAlex
- W1973368209
- Cited by
- 687
- References
- 80
- FWCI
- 17.20
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.