The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review
Open accessAbstract
OBJECTIVE: This review aimed to determine if the use of the patient, intervention, comparison, outcome (PICO) model as a search strategy tool affects the quality of a literature search. METHODS: A comprehensive literature search was conducted in PubMed, Embase, CINAHL, PsycINFO, Cochrane Library, Web of Science, Library and Information Science Abstracts (LISA), Scopus, and the National Library of Medicine (NLM) catalog up until January 9, 2017. Reference lists were scrutinized, and citation searches were performed on the included studies. The primary outcome was the quality of literature searches and the secondary outcome was time spent on the literature search when the PICO model was used as a search strategy tool, compared to the use of another conceptualizing tool or unguided searching. RESULTS: A total of 2,163 records were identified, and after removal of duplicates and initial screening, 22 full-text articles were assessed. Of these, 19 studies were excluded and 3 studies were included, data were extracted, risk of bias was assessed, and a qualitative analysis was conducted. The included studies compared PICO to the PIC truncation or links to related articles in PubMed, PICOS, and sample, phenomenon of interest, design, evaluation, research type (SPIDER). One study compared PICO to unguided searching. Due to differences in intervention, no quantitative analysis was performed. CONCLUSIONS: Only few studies exist that assess the effect of the PICO model vis-a-vis other available models or even vis-a-vis the use of no model. Before implications for current practice can be drawn, well-designed studies are needed to evaluate the role of the tool used to devise a search strategy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The STROBE checklist was used to measure methodological quality of observational studies.
“In this study, the STROBE checklist [17] was used as a tool to measure methodological quality in observational studies.”
From The global prevalence of ADHD in children and adolescents: a systematic review and meta-analysis (2023) · cites this paper as [17]
The cited paper’s Methods describe AMSTAR and PRISMA for the review and self-developed criteria for assessing risk of bias in studies of PICO search strategies. Searches for “STROBE checklist” and its full name found no passage identifying STROBE or addressing its use to measure methodological quality in observational studies. The citing authors’ statement about what they used in their own study cannot be verified from this source. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823 · see citing works
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554 · see citing works
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817 · see citing works
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement
McGowan, J., Sampson, M., Salzwedel, D. M., Cogo, E., Foerster, V., & Lefebvre, C. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.01.021 · Cited by 5,700 · cited 2× · see citing works
“As literature searching forms the underlying basis of systematic reviews, the quality of the literature search is crucially important to the overall quality of the systematic review [3].”
BackgroundMethodMethley, A. M., Campbell, S., Chew-Graham, C., McNally, R., & Cheraghi-Sohi, S. (2014) · BMC Health Services Research · doi:10.1186/s12913-014-0579-0 · Cited by 2,706 · see citing works
Beyond PICO: The SPIDER Tool for Qualitative Evidence Synthesis
Cooke, A., Smith, D., & Booth, A. (2012) · Qualitative Health Research · doi:10.1177/1049732312452938 · Cited by 2,608 · see citing works
Clear and present questions: formulating questions for evidence based practice
Booth, A. (2006) · Library Hi Tech · doi:10.1108/07378830610692127 · Cited by 344 · see citing works
Miller, S. A., & Forrest, J. L. (2001) · Journal of Evidence Based Dental Practice · doi:10.1016/s1532-3382(01)70024-3 · Cited by 339 · cited 1× · see citing works
“The purpose of using PICO is considered to be three-fold [9].”
MethodApplications of text mining within systematic reviews
Thomas, J., McNaught, J., & Ananiadou, S. (2011) · Research Synthesis Methods · doi:10.1002/jrsm.27 · Cited by 212 · see citing works
Flagged first, then by citations
Research works printed but not linked: 14
- [1]Cochrane AL 1931–1971 a critical review, with particular reference to the medical profession Medicines Year 2000 1 11
- [2]Bastian H Glasziou P Chalmers I Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLOS Med 2010 9 21 7 9 e1000326 10.1371/journal.pmed.1000326 20877712 PMC2943439 · DOI · Paper page
- [5]Evidence-Based Medicine Working Group Evidence-based medicine. a new approach to teaching the practice of medicine JAMA 1992 11 4 268 17 2420 5 10.1001/jama.1992.03490170092032 1404801 · DOI
- [6]Richardson WS Wilson MC Nishikawa J Hayward RS The well-built clinical question: a key to evidence-based decisions ACP J Club 1995 Nov-Dec 123 3 A12 3 7582737
- [10]Higgins JP Green S Cochrane handbook for systematic reviews of interventions Wiley Online Library 2008
- [11]Huang X Lin J Demner-Fushman D Evaluation of PICO as a knowledge representation for clinical questions AMIA Annu Symp Proc 2006 359 63 17238363 PMC1839740
- [12]Considine J Shaban RZ Fry M Curtis K Evidence based emergency nursing: designing a research question and searching the literature Int Emerg Nurs 2017 5 32 78 82 10.1016/j.ienj.2017.02.001 28233626 · DOI
- [13]Higgins JPT Green S Cochrane handbook for systematic reviews of interventions Wiley Online Library 2008 Lefebvre CME Glanville J Chapter 6 Searching for studies
- [14]Shea BJ Grimshaw JM Wells GA Boers M Andersson N Hamel C Porter AC Tugwell P Moher D Bouter LM Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews BMC Med Res Methodol 2007 2 15 7 10 10.1186/1471-2288-7-10 17302989 PMC1810543 · DOI · Paper page
- [16]Booth A How much searching is enough? comprehensive versus optimal retrieval for technology assessments Int J Technol Assess Health Care 2010 10 26 4 431 5 10.1017/S0266462310000966 20923586 · DOI
- [18]Buckland M Gey F The relationship between recall and precision J Am Soc Inf Sci 1994 1 45 1 12 10.1002/(SICI)1097-4571(199401)45:1<12::AID-ASI2>3.0.CO;2-L · DOI
- [19]Munn Z Stern C Aromataris E Lockwood C Jordan Z What kind of systematic review should I conduct? a proposed typology and guidance for systematic reviewers in the medical and health sciences BMC Med Res Methodol 2018 1 10 18 1 5 10.1186/s12874-017-0468-4 29316881 PMC5761190 · DOI · Paper page
- [21]Agoritsas T Merglen A Courvoisier DS Combescure C Garin N Perrier A Perneger TV Sensitivity and predictive value of 15 PubMed search strategies to answer clinical questions rated against full systematic reviews J Med Internet Res 2012 6 12 14 3 e85 10.2196/jmir.2021 22693047 PMC3414859 · DOI
- [22]Hoogendam A de Vries Robbe PF Overbeke AJ Comparing patient characteristics, type of intervention, control, and outcome (PICO) queries with unguided searching: a randomized controlled crossover trial J Med Libr Assoc 2012 4 100 2 121 6 10.3163/1536-5050.100.2.010. 10.3163/1536-5050.100.2.010 22514508 PMC3324808 · DOI
Cited by
Sorted by citations
The global prevalence of ADHD in children and adolescents: a systematic review and meta-analysis
Salari, N., Ghasemi, H., Abdoli, N., Rahmani, A., Shiri, M. H., Hashemian, A. H., Akbari, H., & Mohammadi, M. (2023) · Italian Journal of Pediatrics · doi:10.1186/s13052-023-01456-1 · Cited by 670 · see citing works
Multimedia Design for Learning: An Overview of Reviews With Meta-Meta-Analysis
Noetel, M., Griffith, S., Delaney, O., Harris, N. R., Sanders, T., Parker, P., del Pozo Cruz, B., & Lonsdale, C. (2022) · Review of Educational Research · doi:10.3102/00346543211052329 · Cited by 148 · see citing works
“We generated a list of terms that identified all target papers (Eriksen & Frandsen, 2018) optimizing for sensitivity while maintaining specificity.”
Salzman, T., Sarquis-Adamson, Y., Son, S., Montero-Odasso, M., & Fraser, S. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.6744 · Cited by 125 · see citing works
Sebri, V., Durosini, I., Triberti, S., & Pravettoni, G. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.611954 · Cited by 104 · see citing works
Alves, L., Abreo, L., Petkari, E., & Pinto da Costa, M. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.03.081 · Cited by 50 · see citing works
Evidence inference 2.0: More data, better models.
DeYoung, J., Lehman, E., Nye, B., Marshall, I., & Wallace, B. C. (2020) · Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing · doi:10.18653/v1/2020.bionlp-1.13 · Cited by 49 · see citing works
Baena-Marín, M., Rojas-Jaramillo, A., González-Santamaría, J., Rodríguez-Rosell, D., Petro, J. L., Kreider, R. B., & Bonilla, D. A. (2022) · Sports · doi:10.3390/sports10010008 · Cited by 35 · see citing works
Ying, L., Fitzpatrick, J. M., Philippou, J., Huang, W., & Rafferty, A. M. (2021) · Journal of Clinical Nursing · doi:10.1111/jocn.15486 · Cited by 30 · see citing works
Ahorsu, D. K., Adjaottor, E. S., & Lam, B. Y. H. (2021) · Brain Sciences · doi:10.3390/brainsci11070840 · Cited by 23 · see citing works
Review of Hands-On Laboratory Experiments Employing Household Supplies
Salta, K., Ntalakou, E., & Tsiortos, Z. (2022) · Journal of Chemical Education · doi:10.1021/acs.jchemed.2c00037 · Cited by 21 · see citing works
Ilić, J., Ivanović, M., & Klašnja-Milićević, A. (2024) · 2024 21st International Conference on Information Technology Based Higher Education and Training (ITHET) · doi:10.1109/ithet61869.2024.10837649 · Cited by 9 · see citing works
Application of classic grounded theory in nursing studies: a qualitative systematic review protocol
Connor, J., Flenady, T., Dwyer, T., & Massey, D. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-068588 · Cited by 8 · see citing works
Ibrahim, A. A. E., McWilliams, D. F., Smith, S. L., Chaplin, W. J., Salimian, M., Georgopoulos, V., Kouraki, A., & Walsh, D. A. (2025) · Annals of Physical and Rehabilitation Medicine · doi:10.1016/j.rehab.2024.101894 · Cited by 9 · see citing works
Nieto González, D. M. (2022) · doi:10.11144/javeriana.10554.58492 · Cited by 6 · see citing works
Novianti, N. (2024) · Psikologika: Jurnal Pemikiran dan Penelitian Psikologi · doi:10.20885/jstl.vol29.iss2.art7 · Cited by 3 · see citing works
Singh, A., Podder, S., Khairnar, M., & Thakkar, T. (2026) · European Journal of Dental Education · doi:10.1111/eje.70264
- Authors
- Eriksen, M. B., & Frandsen, T. F.
- Affiliations
- University of Southern Denmark (DK)
- Venue
- Journal of the Medical Library Association
- Published
- 2018-10-04
- Volume
- 106
- Issue
- 4
- Pages
- 420-431
- ISSN
- 1536-5050 (ISSN-L)
- 1558-9439
- PMC
- PMC6148624
- OpenAlex
- W2895742152
- Cited by
- 1,216
- References
- 10
- Citations by year
- 2026: 175 · 2025: 271 · 2024: 229 · 2023: 208 · 2022: 165 · 2021: 99 · 2020: 57 · 2019: 11
- FWCI
- 8.24
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Journal of the Medical Library Association JMLA (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- University of Southern Denmark Research Portal (University of Southern Denmark) (acceptedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- search strategy design, PICO, spiders, bibliographic database searching, search time, citation searching
- MeSH terms
- Humans, Periodicals as Topic, Quality Assurance, Health Care, United States, Information Storage and Retrieval, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
