The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Utilization of the PICO framework to improve searching PubMed for clinical questions
Open accessAbstract
Background: Supporting 21 st century health care and the practice of evidence-based medicine (EBM) requires ubiquitous access to clinical information and to knowledge-based resources to answer clinical questions. Many questions go unanswered, however, due to lack of skills in formulating questions, crafting effective search strategies, and accessing databases to identify best levels of evidence. Methods: This randomized trial was designed as a pilot study to measure the relevancy of search results using three different interfaces for the PubMed search system. Two of the search interfaces utilized a specific framework called PICO, which was designed to focus clinical questions and to prompt for publication type or type of question asked. The third interface was the standard PubMed interface readily available on the Web. Study subjects were recruited from interns and residents on an inpatient general medicine rotation at an academic medical center in the US. Thirty-one subjects were randomized to one of the three interfaces, given 3 clinical questions, and asked to search PubMed for a set of relevant articles that would provide an answer for each question. The success of the search results was determined by a precision score, which compared the number of relevant or gold standard articles retrieved in a result set to the total number of articles retrieved in that set. Results: Participants using the PICO templates (Protocol A or Protocol B) had higher precision scores for each question than the participants who used Protocol C, the standard PubMed Web interface. (Question 1: A = 35%, B = 28%, C = 20%; Question 2: A = 5%, B = 6%, C = 4%; Question 3: A = 1%, B = 0%, C = 0%) 95% confidence intervals were calculated for the precision for each question using a lower boundary of zero. However, the 95% confidence limits were overlapping, suggesting no statistical difference between the groups. Conclusion: Due to the small number of searches for each arm, this pilot study could not demonstrate a statistically significant difference between the search protocols. However there was a trend towards higher precision that needs to be investigated in a larger study to determine if PICO can improve the relevancy of search results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The population, interventions, comparisons, outcomes, and study design (PICOS) framework [27] was followed to identify key study concepts in the research question a priori, and to facilitate the search process.”
In the citing paper
“The population, intervention, comparison, and outcome framework [25] was used to develop the search strategy, which was implemented following the PRISMA-P (Preferred Reporting Items for Systematic Review and Meta-Analyses Protocols) checklist [26].”
From The Effectiveness of Artificial Intelligence Conversational Agents in Health Care: Systematic Review (2020) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Evidence-Based Medicine: How to Practice and Teach EBM
Sackett, D. L. (2000) · Cited by 5,950
The well-built clinical question: A key to evidence-based decisions
(1995) · ACP Journal Club · doi:10.7326/acpjc-1995-123-3-a12 · Cited by 2,931
Evidence-based medicine : how to practice and teach EBM
Straus, S. E. (2005) · Cited by 1,367
Evaluation of PICO as a knowledge representation for clinical questions.
Huang, X., Lin, J., & Demner‐Fushman, D. (2006) · Europe PMC (PubMed Central) · Cited by 844
Obstacles to answering doctors' questions about patient care with evidence: qualitative study
Ely, J., Osheroff, J. A., Ebell, M. H., Chambliss, M. L., Vinson, D. C., Stevermer, J. J., & Pifer, E. A. (2002) · BMJ · doi:10.1136/bmj.324.7339.710 · Cited by 502
Green, M. L., & Ruff, T. R. (2005) · Academic Medicine · doi:10.1097/00001888-200502000-00016 · Cited by 246
Do Online Information Retrieval Systems Help Experienced Clinicians Answer Clinical Questions?
Westbrook, J. I. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1717 · Cited by 181
Residents’ medical information needs in clinic: are they being met?
Green, M. L., Ciampi, M., & Ellis, P. J. (2000) · The American Journal of Medicine · doi:10.1016/s0002-9343(00)00458-7 · Cited by 180
Effect of online literature searching on length of stay and patient care costs
Klein, M. S., Ross, F. V., Adams, D. L., & Gilbert, C. M. (1994) · Academic Medicine · doi:10.1097/00001888-199406000-00017 · Cited by 141
Robustness of empirical search strategies for clinical content in MEDLINE.
Wilczynski, N. L., & Haynes, R. B. (2002) · PubMed · Cited by 62
Villanueva, E., Burrows, E., Fennessy, P., Rajendran, M., & Anderson, J. N. (2001) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-1-4 · Cited by 62
Crowley, S. D., Owens, T., Schardt, C. M., Wardell, S., Peterson, J. F., Garrison, S., & Keitz, S. A. (2003) · Academic Medicine · doi:10.1097/00001888-200303000-00007 · Cited by 50
Self-citationSnowball, R. (1997) · Health Libraries Review · doi:10.1046/j.1365-2532.1997.1430133.x-i1 · Cited by 33
Snowball, R. (1997) · Health Libraries Review · doi:10.1046/j.1365-2532.1997.1430167.x · Cited by 27
BERGUS, G. R., & Emerson, M. (2005) · PubMed · Cited by 16
Research works printed but not linked: 1
- PubMed. [http://www.pubmed.gov]
Cited by
Sorted by citations
Beyond PICO: The SPIDER Tool for Qualitative Evidence Synthesis
Cooke, A., Smith, D., & Booth, A. (2012) · Qualitative Health Research · doi:10.1177/1049732312452938 · Cited by 2,608
Poitras, V. J., Gray, C. E., Borghese, M. M., Carson, V., Chaput, J.-P., Janssen, I., Katzmarzyk, P. T., Pate, R. R., Connor Gorber, S., Kho, M. E., et al. (2016) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/apnm-2015-0663 · Cited by 2,217
“The participants, interventions, comparisons, outcomes and study design (PICOS) framework (Schardt et al. 2007) was followed to identify key study concepts in the research question a priori and to facilitate the searching process.”
MethodCarson, V., Hunter, S., Kuzik, N., Gray, C. E., Poitras, V. J., Chaput, J.-P., Saunders, T. J., Katzmarzyk, P. T., Okely, A. D., Connor Gorber, S., et al. (2016) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/apnm-2015-0630 · Cited by 1,380
Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097
Chaput, J.-P., Gray, C. E., Poitras, V. J., Carson, V., Gruber, R., Olds, T., Weiss, S. K., Connor Gorber, S., Kho, M. E., Sampson, M., et al. (2016) · Applied Physiology, Nutrition, and Metabolism · doi:10.1139/apnm-2015-0627 · Cited by 898
Metadata
- Authors
- Schardt, C., Adams, M. B., Owens, T., Keitz, S., & Fontelo, P.
- Affiliations
- Duke University (US)
- Miami VA Healthcare System (US)
- United States National Library of Medicine (US)
- Corresponding author
- Schardt, Connie
- Adams, Martha B
- Owens, Thomas
- Keitz, Sheri
- Fontelo, Paul
- Venue
- BMC Medical Informatics and Decision Making
- Published
- 2007-06-15
- Volume
- 7
- Issue
- 1
- Pages
- 16-16
- ISSN
- 1472-6947 (ISSN-L)
- PMC
- PMC1904193
- OpenAlex
- W2111722073
- Cited by
- 3,053
- References
- 14
- Citations by year
- 2026: 346 · 2025: 460 · 2024: 364 · 2023: 345 · 2022: 380 · 2021: 302 · 2020: 310 · 2019: 134 · 2018: 93 · 2017: 79 · 2016: 57 · 2015: 46 · 2014: 35 · 2013: 18 · 2012: 24
- FWCI
- 24.35
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- PICO framework, PubMed search, clinical question formulation, evidence-based medicine, search result relevance, search interface design, medical information retrieval
- MeSH terms
- Humans, Internship and Residency, Medical Informatics, Probability, Surveys and Questionnaires, Sensitivity and Specificity, MEDLINE, Information Storage and Retrieval, Evidence-Based Medicine, PubMed, Medical Subject Headings
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
