Optimizing literature search in systematic reviews – are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders?
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
BACKGROUND: When conducting systematic reviews, it is essential to perform a comprehensive literature search to identify all published studies relevant to the specific research question. The Cochrane Collaborations Methodological Expectations of Cochrane Intervention Reviews (MECIR) guidelines state that searching MEDLINE, EMBASE and CENTRAL should be considered mandatory. The aim of this study was to evaluate the MECIR recommendations to use MEDLINE, EMBASE and CENTRAL combined, and examine the yield of using these to find randomized controlled trials (RCTs) within the area of musculoskeletal disorders. METHODS: Data sources were systematic reviews published by the Cochrane Musculoskeletal Review Group, including at least five RCTs, reporting a search history, searching MEDLINE, EMBASE, CENTRAL, and adding reference- and hand-searching. Additional databases were deemed eligible if they indexed RCTs, were in English and used in more than three of the systematic reviews. Relative recall was calculated as the number of studies identified by the literature search divided by the number of eligible studies i.e. included studies in the individual systematic reviews. Finally, cumulative median recall was calculated for MEDLINE, EMBASE and CENTRAL combined followed by the databases yielding additional studies. RESULTS: Deemed eligible was twenty-three systematic reviews and the databases included other than MEDLINE, EMBASE and CENTRAL was AMED, CINAHL, HealthSTAR, MANTIS, OT-Seeker, PEDro, PsychINFO, SCOPUS, SportDISCUS and Web of Science. Cumulative median recall for combined searching in MEDLINE, EMBASE and CENTRAL was 88.9% and increased to 90.9% when adding 10 additional databases. CONCLUSION: Searching MEDLINE, EMBASE and CENTRAL was not sufficient for identifying all effect studies on musculoskeletal disorders, but additional ten databases did only increase the median recall by 2%. It is possible that searching databases is not sufficient to identify all relevant references, and that reviewers must rely upon additional sources in their literature search. However further research is needed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE guidelines: 5. Rating the quality of evidence—publication bias
Guyatt, G. H., Oxman, A. D., Montori, V., Vist, G., Kunz, R., Brozek, J., Alonso-Coello, P., Djulbegovic, B., Atkins, D., Falck-Ytter, Y., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.011 · Cited by 1,819
Royle, P., & Milne, R. (2003) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462303000552 · Cited by 693 · cited 3×
“However, they concluded that to fully capture the complete body of available literature on other subjects might require searching multiple databases [20].”
BackgroundMethodAquatic exercise for the treatment of knee and hip osteoarthritis
Bartels, E. M., Lund, H., Hagen, K. B., Dagfinrud, H., Christensen, R., & Danneskiold-Samsøe, B. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005523.pub2 · Cited by 302
Self-citationBramer, W. M., Giustini, D., & Kramer, B. M. R. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0215-7 · Cited by 202
Multidisciplinary rehabilitation for fibromyalgia and musculoskeletal pain in working age adults
Karjalainen, K. A., Malmivaara, A., van Tulder, M. W., Roine, R., Jauhiainen, M., Hurri, H., & Koes, B. W. (1999) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001984 · Cited by 180 · cited 1×
“2010 [43] 2010 Methotrexate monotherapy versus methotrexate combination therapy with non-biologic disease modifying antirheumatic drugs for rheumatoid arthritis (Review) 5 20 MEDLINE, EMBASE, CENTRAL, Medicine”
BackgroundHalladay, C. W., Trikalinos, T. A., Schmid, I. T., Schmid, C. H., & Dahabreh, I. J. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.12.017 · Cited by 166
Should meta-analysts search Embase in addition to Medline?
Sampson, M. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00110-0 · Cited by 160
Searching multiple databases for systematic reviews: added value or diminishing returns?
Stevinson, C., & Lawlor, D. A. (2004) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2004.09.003 · Cited by 143 · cited 1×
“Even though MEDLINE, EMBASE and other major medical databases yield a high proportion of relevant studies, some studies found it necessary to include other sources such as reference- and citation search, browsing conference proceedings, asking experts and alike to identify all the relevant studies [9, 14, 17, 19].”
BackgroundSearching one or two databases was insufficient for meta-analysis of observational studies
Lemeshow, A. R., Blum, R. E., Berlin, J. A., Stoto, M. A., & Colditz, G. A. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.03.004 · Cited by 137
Michaleff, Z. A., Costa, L. O. P., Moseley, A. M., Maher, C. G., Elkins, M. R., Herbert, R. D., & Sherrington, C. (2011) · Physical Therapy · doi:10.2522/ptj.20100116 · Cited by 108
Research works printed but not linked: 65
- Gray JAM, Booth A, Booth S, Ison E. Evidence-based healthcare. Edinburgh: Churchill Livingstone; 2001.
- Lund H, Brunnhuber K, Juhl C, Robinson K, Leenaars M, Dorch BF, Jamtvedt G, Nortvedt MW, Christensen R, Chalmers I. Towards evidence based research. BMJ. 2016;355:i5440. · DOI · Paper page
- Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Chichester: John Wiley & Sons; 2008. · DOI · Paper page
- Chalmers I, Nylenna M. A new network to promote evidence-based research. Lancet. 2014;384(9958):1903–4. · DOI
- Klassen TP, Jadad AR, Moher D. Guides for reading and interpreting systematic reviews: I. Getting started. Arch Pediatr Adolesc Med. 1998;152(7):700–4. · DOI
- Chandler J, Churchill R, Higgins J, Lasserson T, Tovey D. Methodological standards for the conduct of new Cochrane Intervention Reviews. The Cochrane Unit. 2013;2:3.
- Rollin L, Darmoni S, Caillard JF, Gehanno JF. Searching for high-quality articles about intervention studies in occupational health - What is really missed when using only the Medline database? Scand J Work Environ Health. 2010;36(6):484–7. · DOI · Paper page
- Murphy LS, Reinsch S, Najm WI, Dickerson VM, Seffinger MA, Adams A, Mishra SI. Spinal palpation: The challenges of information retrieval using available databases. J Manipulative Physiol Ther. 2003;26(6):374–82. · doi:10.1016/s0161-4754(03)00076-9
- Sampson M, Cogo E, Ajiferuke I, Manheimer E, Campbell K, Daniel R, Moher D. Searching for controlled trials of complementary and alternative medicine: A comparison of 15 databases. Evid Based Complement Alternat Med. 2011;2011:858246. · DOI
- Shariff SZ, Sontrop JM, Iansavichus AV, Haynes RB, Weir MA, Gandhi S, Cuerden MS, Garg AX. Availability of renal literature in six bibliographic databases. Clin Kidney J. 2012;5(6):610–7. · DOI
- Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic Databases: MEDLINE alone is not enough. Control Clin Trials. 2000;21(5):476–87. · DOI · Paper page
- Vickers AJ. Bibliometric analysis of randomized trials in complementary medicine. Complement Ther Med. 1998;6(4):185–9. · DOI
- Whiting P, Westwood M, Burke M, Sterne J, Glanville J. Systematic reviews of test accuracy should search a range of databases to identify primary studies. J Clin Epidemiol. 2008;61(4):357. · DOI
- Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health. 2005;59(9):804–8. · DOI · Paper page
- Woolf AD, Pfleger B. Burden of major musculoskeletal conditions. Bull World Health Organ. 2003;81(9):646–56.
- Horton R. GBD 2010: understanding disease, injury, and risk. Lancet. 2012;380(9859):2053–4. · DOI · Paper page
- Utterback DF, Schnorr TM: Use of workers’ compensation data for occupational safety and health: proceedings from June 2012 workshop. In US Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. In. http://www.cdc.gov/niosh/topics/workercomp/cwcs/publications.html : Assessed 25 Aug 2016.
- Harter SP. Online information retrieval : concepts, principles and techniques. Orlando: Academic; 1986.
- Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, Klassen TP, Moher D. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006;6:33. · DOI · Paper page
- Vincent S, Greenley S, Beaven O. Clinical Evidence diagnosis: Developing a sensitive search strategy to retrieve diagnostic studies on deep vein thrombosis: a pragmatic approach. Health Info Libr J. 2003;20(3):150–9. · DOI · Paper page
- Watson RJD, Richardson PH. Identifying randomized controlled trials of cognitive therapy for depression: Comparing the efficiency of embase, medline and PsycINFO bibliographic databases. Psychol Psychother Theory Res Pract. 1999;72(4):535–42.
- Adie S, Kwan A, Naylor JM, Harris IA, Mittal R. Cryotherapy following total knee replacement. Cochrane Database Syst Rev. 2012;9, CD007911.
- Bellamy N, Campbell J, Robinson V, Gee T, Bourne R, Wells G. Viscosupplementation for the treatment of osteoarthritis of the knee. Cochrane Database Syst Rev. 2006;2, CD005321.
- Bellamy N, Campbell J, Robinson V, Gee T, Bourne R, Wells G. Intraarticular corticosteroid for treatment of osteoarthritis of the knee. Cochrane Database Syst Rev. 2006;2, CD005328.
- Coghlan JA, Buchbinder R, Green S, Johnston RV, Bell SN. Surgery for rotator cuff disease. Cochrane Database Syst Rev. 2008;1, CD005619.
- Colebatch AN, Marks JL, Edwards CJ. Safety of non-steroidal anti-inflammatory drugs, including aspirin and paracetamol (acetaminophen) in people receiving methotrexate for inflammatory arthritis (rheumatoid arthritis, ankylosing spondylitis, psoriatic arthritis, other spondyloarthritis). Cochrane Database Syst Rev. 2011;11, CD008872.
- Cranney A, Welch V, Adachi JD, Homik J, Shea B, Suarez-Almazor ME, Tugwell P, Wells G. Calcitonin for the treatment and prevention of corticosteroid-induced osteoporosis. Cochrane Database Syst Rev. 2000;2, CD001983.
- De Morton NA, Keating JL, Jeffs K. Exercise for acutely hospitalised older medical patients. Cochrane Database Syst Rev. 2007;1, CD005955.
- Fidelix TS, Soares BG, Trevisani VF. Diacerein for osteoarthritis. Cochrane Database Syst Rev. 2006;1, CD005117.
- Katchamart W, Trudeau J, Phumethum V, Bombardier C. Methotrexate monotherapy versus methotrexate combination therapy with non-biologic disease modifying anti-rheumatic drugs for rheumatoid arthritis. Cochrane Database Syst Rev. 2010;4, CD008495.
- Khan F, Ng L, Gonzalez S, Hale T, Turner-Stokes L. Multidisciplinary rehabilitation programmes following joint replacement at the hip and knee in chronic arthropathy. Cochrane Database Syst Rev. 2008;2, CD004957.
- Lethaby A, Lopez-Olivo MA, Maxwell L, Burls A, Tugwell P, Wells GA. Etanercept for the treatment of rheumatoid arthritis. Cochrane Database Syst Rev. 2013;5, CD004525.
- Nuesch E, Rutjes AW, Husni E, Welch V, Juni P. Oral or transdermal opioids for osteoarthritis of the knee or hip. Cochrane Database Syst Rev. 2009;4, CD003115.
- Osiri M, Shea B, Robinson V, Suarez-Almazor M, Strand V, Tugwell P, Wells G. Leflunomide for treating rheumatoid arthritis. Cochrane Database Syst Rev. 2003;1, CD002047.
- Richards BL, Whittle SL, Buchbinder R. Antidepressants for pain management in rheumatoid arthritis. Cochrane Database Syst Rev. 2011;11, CD008920.
- Richards BL, Whittle SL, Buchbinder R. Muscle relaxants for pain management in rheumatoid arthritis. Cochrane Database Syst Rev. 2012;1, CD008922.
- Ruiz Garcia V, Jobanputra P, Burls A, Cabello JB, Galvez Munoz JG, Saiz Cuenca ES, Fry-Smith A. Certolizumab pegol (CDP870) for rheumatoid arthritis in adults. Cochrane Database Syst Rev. 2011;2, CD007649.
- Rutjes AW, Nuesch E, Sterchi R, Juni P. Therapeutic ultrasound for osteoarthritis of the knee or hip. Cochrane Database Syst Rev. 2010;1, CD003132.
- Rutjes AW, Nuesch E, Sterchi R, Kalichman L, Hendriks E, Osiri M, Brosseau L, Reichenbach S, Juni P. Transcutaneous electrostimulation for osteoarthritis of the knee. Cochrane Database Syst Rev. 2009;4, CD002823.
- Wajon A, Ada L, Edmunds I. Surgery for thumb (trapeziometacarpal joint) osteoarthritis. Cochrane Database Syst Rev. 2005;4, CD004631.
- Whittle SL, Richards BL, Husni E, Buchbinder R. Opioid therapy for treating rheumatoid arthritis pain. Cochrane Database Syst Rev. 2011;11, CD003113.
- Winzenberg TM, Powell S, Shaw KA, Jones G. Vitamin D supplementation for improving bone mineral density in children. Cochrane Database Syst Rev. 2010;10, CD006944.
- Brosseau L, Judd MG, Marchand S, Robinson VA, Tugwell P, Wells G, Yonge K. Transcutaneous electrical nerve stimulation (TENS) for the treatment of rheumatoid arthritis in the hand. Cochrane Database Syst Rev. 2003;3, CD004377.
- O’Donnell S, Cranney A, Wells GA, Adachi JD, Reginster JY. Strontium ranelate for preventing and treating postmenopausal osteoporosis. Cochrane Database Syst Rev. 2006;3, CD005326.
- Richards BL, Whittle SL, Buchbinder R. Neuromodulators for pain management in rheumatoid arthritis. Cochrane Database Syst Rev. 2012;1, CD008921.
- Rome K, Ashford RL, Evans A. Non-surgical interventions for paediatric pes planus. Cochrane Database Syst Rev. 2010;7, CD006311.
- Savoie I, Helmer D, Green CJ, Kazanjian A. Beyond Medline: reducing bias through extended systematic review search. Int J Technol Assess Health Care. 2003;19(1):168–78. · DOI · Paper page
- Helmer D, Savoie I, Green C, Kazanjian A. Evidence-based practice: extending the search to find material for the systematic review. Bull Med Libr Assoc. 2001;89(4):346–52.
- Robinson KA, Dunn AG, Tsafnat G, Glasziou P. Citation networks of related trials are often disconnected: implications for bidirectional citation searches. J Clin Epidemiol. 2014;67(7):793–9. · DOI
- Beckles Z, Glover S, Ashe J, Stockton S, Boynton J, Lai R, Alderson P. Searching CINAHL did not add value to clinical questions posed in NICE guidelines. J Clin Epidemiol. 2013;66(9):1051–7. · DOI
- Kelly L, St Pierre-Hansen N. So many databases, such little clarity: Searching the literature for the topic aboriginal. Can Fam Physician. 2008;54(11):1572–1573.e1575.
- Adams CE, Power A, Frederick K, Lefebvre C. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med. 1994;24(3):741–8. · DOI · Paper page
- Aker PD, McDermaid C, Opitz BG, White MW. Searching chiropractic literature: A comparison of three computerized databases. J Manip Physiol Ther. 1996;19(8):518–24.
- Crumley ET, Wiebe N, Cramer K, Klassen TP, Hartling L. Which resources should be used to identify RCT/CCTs for systematic reviews: a systematic review. BMC Med Res Methodol. 2005;5:24. · DOI · Paper page
- Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ. 1994;309(6964):1286–91. · DOI · Paper page
- Haafkens J, Moerman C, Schuring M, Van Dijk F. Searching bibliographic databases for literature on chronic disease and work participation. Occup Med. 2006;56(1):39–45. · DOI · Paper page
- Li L, Tian J, Tian H, Moher D, Liang F, Jiang T, Yao L, Yang K. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. 2014;67(9):1001–7. · DOI · Paper page
- Lovarini M, Wallen M, Imms C. Searching for evidence in pediatric occupational therapy using free versus subscription databases: A comparison of outcomes. Phys Occup Ther Pediatr. 2006;26(3):19–38. · DOI
- Matthews EJ, Edwards AG, Barker J, Bloor M, Covey J, Hood K, Pill R, Russell I, Stott N, Wilkinson C. Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care. Health Libr Rev. 1999;16(2):112–20. · DOI · Paper page
- Royle PL, Bain L, Waugh NR. Sources of evidence for systematic reviews of interventions in diabetes. Diabet Med. 2005;22(10):1386–93. · DOI · Paper page
- Woodman J, Harden A, Thomas J, Brunton J, Kavanagh J, Stansfield C. Searching for systematic reviews of the effects of social and environmental interventions: a case study of children and obesity. J Med Libr Assoc. 2010;98(2):140–6. · DOI
- Wright JM, Cottrell DJ, Mir G. Searching for religion and mental health studies required health, social science, and grey literature databases. J Clin Epidemiol. 2014;67(7):800–10. · DOI
- Ramos-Remus C, Suarez-Almazor M, Dorgan M, Gomez-Vargas A, Russell AS. Performance of online biomedical databases in rheumatology. J Rheumatol. 1994;21(10):1912–21.
- Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62(9):944–52. · DOI · Paper page
- Schoones JW, Vissers T. Re: “using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions”. J Clin Epidemiol. 2016. · DOI
Cited by
Bramer, W. M., Rethlefsen, M. L., Kleijnen, J., & Franco, O. H. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0644-y · Cited by 2,114
Cooper, C., Booth, A., Varley-Campbell, J., Britten, N., & Garside, R. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0545-3 · Cited by 561
Searching two or more databases decreased the risk of missing relevant studies: a metaresearch study
Ewald, H., Klerings, I., Wagner, G., Heise, T. L., Stratil, J. M., Lhachimi, S. K., Hemkens, L. G., Gartlehner, G., Armijo-Olivo, S., & Nussbaumer-Streit, B. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.05.022 · Cited by 168
Frandsen, T. F., Eriksen, M. B., Hammer, D. M. G., & Christensen, J. B. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.04.015 · Cited by 44
Metadata
- Authors
- Aagaard, T., Lund, H., & Juhl, C.
- Venue
- BMC Medical Research Methodology
- Published
- 2016-11-22
- Volume
- 16
- Issue
- 1
- Pages
- 161-161
- ISSN
- 1471-2288 (ISSN-L)
- OpenAlex
- W2552106399
- Cited by
- 66
- References
- 17
- FWCI
- 1.74
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Optimizing literature search in systematic reviews – are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders? https://app.cochat.ai/papers/10.1186/s12874-016-0264-6
- The paper
- Aagaard, T., Lund, H., & Juhl, C. (2016). Optimizing literature search in systematic reviews – are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders? BMC Medical Research Methodology. https://doi.org/10.1186/s12874-016-0264-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.