The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Association Between Funding and Quality of Published Medical Education Research
Abstract
CONTEXT: Methodological shortcomings in medical education research are often attributed to insufficient funding, yet an association between funding and study quality has not been established. OBJECTIVES: To develop and evaluate an instrument for measuring the quality of education research studies and to assess the relationship between funding and study quality. DESIGN, SETTING, AND PARTICIPANTS: Internal consistency, interrater and intrarater reliability, and criterion validity were determined for a 10-item medical education research study quality instrument (MERSQI). This was applied to 210 medical education research studies published in 13 peer-reviewed journals between September 1, 2002, and December 31, 2003. The amount of funding obtained per study and the publication record of the first author were determined by survey. MAIN OUTCOME MEASURES: Study quality as measured by the MERSQI (potential maximum total score, 18; maximum domain score, 3), amount of funding per study, and previous publications by the first author. RESULTS: The mean MERSQI score was 9.95 (SD, 2.34; range, 5-16). Mean domain scores were highest for data analysis (2.58) and lowest for validity (0.69). Intraclass correlation coefficient ranges for interrater and intrarater reliability were 0.72 to 0.98 and 0.78 to 0.998, respectively. Total MERSQI scores were associated with expert quality ratings (Spearman rho, 0.73; 95% confidence interval [CI], 0.56-0.84; P < .001), 3-year citation rate (0.8 increase in score per 10 citations; 95% CI, 0.03-1.30; P = .003), and journal impact factor (1.0 increase in score per 6-unit increase in impact factor; 95% CI, 0.34-1.56; P = .003). In multivariate analysis, MERSQI scores were independently associated with study funding of $20 000 or more (0.95 increase in score; 95% CI, 0.22-1.86; P = .045) and previous medical education publications by the first author (1.07 increase in score per 20 publications; 95% CI, 0.15-2.23; P = .047). CONCLUSION: The quality of published medical education research is associated with study funding.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646
(1991) · Cited by 15,554
Des Jarlais, D. C., Lyles, C., Crepaz, N., & the TREND Group (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.361 · Cited by 2,817
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
Moher, D. (2001) · JAMA · doi:10.1001/jama.285.15.1987 · Cited by 2,545
Pharmaceutical industry sponsorship and research outcome and quality: systematic review
Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003) · BMJ · doi:10.1136/bmj.326.7400.1167 · Cited by 2,147
Current Concepts in Validity and Reliability for Psychometric Instruments: Theory and Application
Cook, D. A., & Beckman, T. J. (2006) · The American Journal of Medicine · doi:10.1016/j.amjmed.2005.10.036 · Cited by 1,745
Self-citationCurrent methods of the U.S. Preventive Services Task Force
Harris, R. P., Helfand, M., Woolf, S. H., Lohr, K. N., Mulrow, C. D., Teutsch, S. M., & Atkins, D. (2001) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(01)00261-6 · Cited by 1,731
Validity: on the meaningful interpretation of assessment data
Downing, S. M. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01594.x · Cited by 1,434
Systems to rate the strength of scientific evidence.
West, S. L., King, V., Carey, T. S., Lohr, K. N., Mckoy, N. A., Sutton, S. F., & Lux, L. J. (2002) · PubMed · Cited by 923
Reeves, M. (1993) · Cited by 903
Als-Nielsen, B., Chen, W., Gluud, C., & Kjaergard, L. L. (2003) · JAMA · doi:10.1001/jama.290.7.921 · Cited by 746
Relative Citation Impact of Various Study Designs in the Health Sciences
Patsopoulos, N. A. (2005) · JAMA · doi:10.1001/jama.293.19.2362 · Cited by 498
The uncertainty principle and industry-sponsored research
Djulbegovic, B., Lacevic, M., Cantor, A., Fields, K. K., Bennett, C. L., Adams, J. R., Kuderer, N. M., & Lyman, G. H. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02605-2 · Cited by 482
BEME Guide No. 1: Best Evidence Medical Education
M. Harden, J. G. (1999) · Medical Teacher · doi:10.1080/01421599978960 · Cited by 353
The Quality of Drug Studies Published in Symposium Proceedings
Cho, M. K., & Bero, L. A. (1996) · Annals of Internal Medicine · doi:10.7326/0003-4819-124-5-199603010-00004 · Cited by 328
Appraising qualitative research articles in medicine and medical education
Côté, L., & Turgeon, J. (2005) · Medical Teacher · doi:10.1080/01421590400016308 · Cited by 312
Association of Journal Quality Indicators With Methodological Quality of Clinical Research Articles
Lee, K. P. (2002) · JAMA · doi:10.1001/jama.287.21.2805 · Cited by 284
Bordage, G. (2001) · Academic Medicine · doi:10.1097/00001888-200109000-00010 · Cited by 235
A National Survey of Policies on Disclosure of Conflicts of Interest in Biomedical Research
McCrary, S. V., Anderson, C. B., Jakovljevic, J., Khan, T., McCullough, L. B., Wray, N. P., & Brody, B. A. (2000) · New England Journal of Medicine · doi:10.1056/nejm200011303432207 · Cited by 225
Research in Medical Education: Balancing Service and Science*
Albert, M., Hodges, B., & Regehr, G. (2007) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9026-2 · Cited by 212
A Call for Outcomes Research in Medical Education
Chen, F. M., Bauchner, H., & Burstin, H. (2004) · Academic Medicine · doi:10.1097/00001888-200410000-00010 · Cited by 205
RCT = results confounded and trivial: the perils of grand educational experiments
Norman, G. (2003) · Medical Education · doi:10.1046/j.1365-2923.2003.01586.x · Cited by 197
What is the validity evidence for assessments of clinical teaching?
Beckman, T. J., Cook, D. A., & Mandrekar, J. N. (2005) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2005.0258.x · Cited by 195
Self-citationCounting on citations: a flawed way to measure quality
Walter, G., Fisher, K., Bloch, S., & Hunt, G. (2003) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2003.tb05196.x · Cited by 166
Advancing Scientific Research in Education
National Research Council (2004) · National Academies Press eBooks · doi:10.17226/11112 · Cited by 155
Prystowsky, J. B., & Bordage, G. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.00910.x · Cited by 147
Carney, P. A., Nierenberg, D. W., Pipas, C. F., Brooks, W. B., Stukel, T. A., & Keller, A. M. (2004) · JAMA · doi:10.1001/jama.292.9.1044 · Cited by 132
Trends in Medical Education Research
Regehr, G. (2004) · Academic Medicine · doi:10.1097/00001888-200410000-00008 · Cited by 131
Costs and Funding for Published Medical Education Research
Reed, D. A., Kern, D. E., Levine, R. B., & Wright, S. M. (2005) · JAMA · doi:10.1001/jama.294.9.1052 · Cited by 128
Self-citationRandomized clinical trials in Hepatology: Predictors of quality
Gluud, L. L., Nikolova, D., & Gluud, C. N. (1999) · Hepatology · doi:10.1002/hep.510300510 · Cited by 118
Funding source, trial outcome and reporting quality: are they related? Results of a pilot study
Clifford, T., Barrowman, N. J., & Moher, D. (2002) · BMC Health Services Research · doi:10.1186/1472-6963-2-18 · Cited by 115
Measuring effectiveness for best evidence medical education: a discussion
Belfield, C., Thomas, H., Bullock, A. D., Eynon, R., & Wall, D. (2001) · Medical Teacher · doi:10.1080/0142150020031084 · Cited by 112
Researching the outcomes of educational interventions: a matter of design
Prideaux, D. J. (2002) · BMJ · doi:10.1136/bmj.324.7330.126 · Cited by 108
Academic Health Centers: Leading Change in the 21st Century
Porter, J. E., Aiken, L. H., Bennett, J. C., Bienen, H., DeParle, N.-A., Holmes, E., Lewin, L., Lurie, N., Paul, S., Ramsey, P. G., et al. (2004) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2004.tb00760.x · Cited by 105
Dauphinée, W. D., & Wood-Dauphinée, S. (2004) · Academic Medicine · doi:10.1097/00001888-200410000-00005 · Cited by 97
Funding Medical Education Research: Opportunities and Issues
Carline, J. D. (2004) · Academic Medicine · doi:10.1097/00001888-200410000-00004 · Cited by 91
The accountability of clinical education: its definition and assessment
Murray, E., Gruppen, L. D., Catton, P. A., Hays, R., & Woolliscroft, J. O. (2000) · Medical Education · doi:10.1046/j.1365-2923.2000.00757.x · Cited by 88
A Pilot Randomized, Controlled Trial of a Longitudinal Residents-as-teachers Curriculum
Morrison, E., Rucker, L., Boker, J. R., Hollingshead, J. R., Hitchcock, M. A., Prislin, M. D., & Hubbell, F. A. (2003) · Academic Medicine · doi:10.1097/00001888-200307000-00016 · Cited by 78
Raising the Passing Grade for Studies of Medical Education
Lurie, S. J. (2003) · JAMA · doi:10.1001/jama.290.9.1210 · Cited by 76
Whitcomb, M. E. (2002) · Academic Medicine · doi:10.1097/00001888-200211000-00001 · Cited by 74
Morrison, J. M., Sullivan, F., Murray, E., & Jolly, B. C. (1999) · Medical Education · doi:10.1046/j.1365-2923.1999.00479.x · Cited by 72
Research in medical education: asking the right questions
Prideaux, D. J., & Bligh, J. G. (2002) · Medical Education · doi:10.1046/j.1365-2923.2002.01411.x · Cited by 72
Guidelines for evaluating papers on educational interventions
Education Group for Guidelines on Evaluation (1999) · BMJ · doi:10.1136/bmj.318.7193.1265 · Cited by 72
A RIME Perspective on the Quality and Relevance of Current and Future Medical Education Research
Shea, J. A., Arnold, L., & Mann, K. V. (2004) · Academic Medicine · doi:10.1097/00001888-200410000-00006 · Cited by 70
Identifying, Appraising, and Implementing Medical Education Curricula: A Guide for Medical Educators
Green, M. L. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-10-200111200-00009 · Cited by 61
The importance of clinical outcomes in medical education research
Chen, F. M., Burstin, H. R., & Huntington, J. (2005) · Medical Education · doi:10.1111/j.1365-2929.2005.02117.x · Cited by 40
Revisiting the Idea of a National Center for Health Professions Education Research
Wartman, S. A. (2004) · Academic Medicine · doi:10.1097/00001888-200410000-00003 · Cited by 32
Assessing the evidence in qualitative medical education research
Stacy, R., & Spencer, J. (2000) · Medical Education · doi:10.1046/j.1365-2923.2000.00710.x · Cited by 28
Graduate Medical Education Research in the 21st Century and JAMA On Call
Lim, J. K., & Golub, R. M. (2004) · JAMA · doi:10.1001/jama.292.23.2913 · Cited by 27
Funding, disease area, and internal validity of hepatobiliary randomized clinical trials
Kjaergard, L., & Gluud, C. (2002) · American Journal of Gastroenterology · doi:10.1111/j.1572-0241.2002.07067.x · Cited by 24
Impact Factor as Measure of Scientific Quality
Semenzato, G., Rizzato, G., Agostini, C., & Tobin, M. J. (2004) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/ajrccm.169.9.950 · Cited by 20
Researching outcomes of educational interventions
Fitz‐Gibbon, C. T. (2002) · BMJ · doi:10.1136/bmj.324.7346.1155/a · Cited by 5
Cited by
Cook, D. A., Hatala, R., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., Erwin, P. J., & Hamstra, S. J. (2011) · JAMA · doi:10.1001/jama.2011.1234 · Cited by 2,170
Self-citationA review of the use of virtual reality head-mounted displays in education and training
Jensen, L., & Konradsen, F. (2018) · Education and Information Technologies · doi:10.1007/s10639-017-9676-0 · Cited by 1,638
Internet-Based Learning in the Health Professions: A Meta-analysis
Cook, D. A., Levinson, A. J., Garside, S., Dupras, D. M., Erwin, P. J., & Montori, V. M. (2008) · JAMA · doi:10.1001/jama.300.10.1181 · Cited by 1,498
Self-citationFlipped classroom improves student learning in health professions education: a meta-analysis
HEW, K. F., & LO, C. K. (2018) · BMC Medical Education · doi:10.1186/s12909-018-1144-z · Cited by 1,207
Hamilton, D., McKechnie, J., Edgerton, E., & Wilson, C. (2021) · Journal of Computers in Education · doi:10.1007/s40692-020-00169-2 · Cited by 1,081
Metadata
- Authors
- Reed, D. A., Cook, D. A., Beckman, T. J., Levine, R. B., Kern, D. E., & Wright, S. M.
- Venue
- JAMA
- Publisher
- American Medical Association (AMA)
- Published
- 2007-09-05
- Volume
- 298
- Issue
- 9
- Pages
- 1002
- ISSN
- 0002-9955 (ISSN-L)
- 0098-7484
- OpenAlex
- W2076134402
- Cited by
- 958
- References
- 54
- FWCI
- 13.76
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.