Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.

Reproducibility of Literature Search Reporting in Medical Education Reviews

Abstract

PURPOSE: Medical education literature has been found to lack key components of scientific reporting, including adequate descriptions of literature searches, thus preventing medical educators from replicating and building on previous scholarship. The purpose of this study was to examine the reproducibility of search strategies as reported in medical education literature reviews. METHOD: The authors searched for and identified literature reviews published in 2009 in Academic Medicine, Teaching and Learning in Medicine, and Medical Education. They searched for citations whose titles included the words "meta-analysis," "systematic literature review," "systematic review," or "literature review," or whose publication type MEDLINE listed as "meta-analysis" or "review." The authors created a checklist to identify key characteristics of literature searches and of literature search reporting within the full text of the reviews. The authors deemed searches reproducible only if the review reported both a search date and Boolean operators. RESULTS: Of the 34 reviews meeting the inclusion criteria, 19 (56%) explicitly described a literature search and mentioned MEDLINE; however, only 14 (41%) also mentioned searches of nonmedical databases. Eighteen reviews (53%) listed search terms, but only 6 (18%) listed Medical Subject Headings, and only 2 (6%) mentioned Boolean operators. Fifteen (44%) noted the use of limits. None of the reviews included reproducible searches. CONCLUSIONS: According to this analysis, literature search strategies in medical education reviews are highly variable and generally not reproducible. The authors provide recommendations to facilitate future high-quality, transparent, and reproducible literature searches.

Overview

Integrity signals

Registry

Crossref · 10.1097/acm.0b013e31822221e7

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 28 references

Research works printed but not linked: 3

  1. Haig, 2003, Med Teach, 25, 352
  2. McGowan, 2005, J Med Libr Assoc, 93, 74
  3. 1997, JAMA, 277, 927, 10.1001/jama.277.11.927 · doi:10.1001/jama.277.11.927 · Paper page

Cited by

See all 77

Metadata

Authors
Maggio, L. A., Tannery, N. H., & Kanter, S. L.
Affiliations
  • Stanford Medicine (US)
  • Stanford University (US)
Corresponding author
Maggio, Lauren A.
Venue
Academic Medicine
Published
2011-08
Volume
86
Issue
8
Pages
1049-1054
ISSN
1040-2446 (ISSN-L)
OpenAlex
W1992951084
Cited by
77
References
28
Citations by year
2026: 3 · 2025: 2 · 2024: 7 · 2023: 8 · 2022: 4 · 2021: 14 · 2020: 7 · 2019: 2 · 2018: 4 · 2017: 3 · 2016: 5 · 2015: 4 · 2014: 5 · 2013: 5 · 2012: 4
FWCI
4.77
Type
journal article
Language
en
Open access
closed
Locations
  • Academic Medicine (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Library and Information Sciences › Academic Writing and Publishing
  • Social Sciences › Education › Innovations in Medical Education
Keywords
reproducibility, search strategies, medical education research, systematic reviews, meta-analysis, Boolean operators, Medical Subject Headings, search limits, research transparency
MeSH terms
Education, Medical, Humans, Review Literature as Topic, Reproducibility of Results, Information Storage and Retrieval

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Reproducibility of Literature Search Reporting in Medical Education Reviews. https://app.cochat.ai/papers/10.1097/acm.0b013e31822221e7
The paper
Maggio, L. A., Tannery, N. H., & Kanter, S. L. (2011). Reproducibility of Literature Search Reporting in Medical Education Reviews. Academic Medicine. https://doi.org/10.1097/acm.0b013e31822221e7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.