The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Systematic Literature Reviews: An Introduction
Open accessAbstract
Systematic literature reviews (SRs) are a way of synthesising scientific evidence to answer a particular research question in a way that is transparent and reproducible, while seeking to include all published evidence on the topic and appraising the quality of this evidence. SRs have become a major methodology in disciplines such as public policy research and health sciences. Some have advocated that design research should adopt the method. However, little guidance is available. This paper provides an overview of the SR method, based on the literature in health sciences. Then, the rationale for SRs in design research is explored, and four recent examples of SRs in design research are analysed to illustrate current practice. Foreseen challenges in taking forward the SR method in design research are highlighted, and directions for developing a SR method for design research are proposed. It is concluded that SRs hold potential for design research and could help us in addressing some important issues, but work is needed to define what review methods are appropriate for each type of research question in design research, and to adapt guidance to our own needs and specificities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769
Tranfield, D., Denyer, D., & Smart, P. (2003) · British Journal of Management · doi:10.1111/1467-8551.00375 · Cited by 15,386
A typology of reviews: an analysis of 14 review types and associated methodologies
Grant, M. J., & Booth, A. (2009) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2009.00848.x · Cited by 10,817
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537
(1997) · Cited by 4,204
Yeh, M.-L. (2014) · Journal of Nursing Research · doi:10.1097/jnr.0000000000000065 · Cited by 3,829
Systematic Reviews in Health Care : Meta-Analysis in Context
Egger, M., Smith, G. D., & Altman, D. G. (2000) · Cited by 2,401
Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions
Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-126-5-199703010-00006 · Cited by 2,184
Synthesising qualitative and quantitative evidence: A review of possible methods
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton, A. (2005) · Journal of Health Services Research & Policy · doi:10.1177/135581960501000110 · Cited by 2,132
Greenhalgh, T., & Peacock, R. (2005) · BMJ · doi:10.1136/bmj.38636.593461.68 · Cited by 2,045
DRM, a Design Research Methodology
Blessing, L. T. M., & Chakrabarti, A. (2009) · Springer eBooks · doi:10.1007/978-1-84882-587-1 · Cited by 1,643
Murad, M. H., Asi, N., Alsawas, M., & Alahdab, F. (2016) · Evidence Based Medicine · doi:10.1136/ebmed-2016-110401 · Cited by 1,388
Systematic Reviews in Health Care. Meta-analysis in Context.
Kleijnen, J., & Antes, G. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.3.697 · Cited by 1,363
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188
Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0468-4 · Cited by 1,164
A taxonomy of circular economy indicators
Saidani, M., Yannou, B., Leroy, Y., Cluzel, F., & Kendall, A. (2019) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2018.10.014 · Cited by 1,093
Clarifying differences between review designs and methods
Gough, D., Thomas, J., & Oliver, S. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-28 · Cited by 1,063
The Medical Review Article: State of the Science
MULROW, C. D. (1987) · Annals of Internal Medicine · doi:10.7326/0003-4819-106-3-485 · Cited by 856
How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis
Shojania, K. G., Sampson, M., Ansari, M. T., Ji, J., Doucette, S., & Moher, D. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-4-200708210-00179 · Cited by 838 · cited 1×
“Indeed, research continues to be published after the search strategy has been completed, and the results of many SRs become quickly outdated by new publications (Shojania et al., 2007).”
BackgroundChassin, M. R. (1987) · JAMA · doi:10.1001/jama.258.18.2533 · Cited by 655
A brief history of research synthesis
Chalmers, I., Hedges, L. V., & Cooper, H. (2002) · Evaluation & the Health Professions · doi:10.1177/0163278702025001003 · Cited by 552
Systematic Reviews from Astronomy to Zoology: Myths and Misconceptions
Petticrew, M. (2001) · BMJ · doi:10.1136/bmj.322.7278.98 · Cited by 550
Can We Systematically Review Studies That Evaluate Complex Interventions?
Shepperd, S., Lewin, S., Straus, S., Clarke, M., Eccles, M. P., Fitzpatrick, R., Wong, G., & Sheikh, A. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000086 · Cited by 408
Article 50 million: an estimate of the number of scholarly articles in existence
JINHA, A. E. (2010) · Learned Publishing · doi:10.1087/20100308 · Cited by 404
Rethlefsen, M. L., Farrell, A. M., Osterhaus Trzasko, L. C., & Brigham, T. J. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.11.025 · Cited by 331
van Aken, J. E., & Romme, G. (2009) · Organization Management Journal · doi:10.1057/omj.2009.1 · Cited by 265
A review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed
Crowe, M., & Sheppard, L. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.02.008 · Cited by 261
How evidence‐based medicine is failing due to biased trials and selective publication
Every‐Palmer, S., & Howick, J. (2014) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12147 · Cited by 223
Bramer, W. M., Giustini, D., & Kramer, B. M. R. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0215-7 · Cited by 202 · cited 1×
“Reviews of published SRs have found that only around 2% of the abstracts screened for the review are ultimately included (Bramer et al., 2016).”
BackgroundFixation or inspiration? A meta-analytic review of the role of examples on design processes
Sio, U. N., Kotovsky, K., & Cagan, J. (2015) · Design Studies · doi:10.1016/j.destud.2015.04.004 · Cited by 191
A systematic literature review on modular product design
Bonvoisin, J., Halstenberg, F., Buchert, T., & Stark, R. (2016) · Journal of Engineering Design · doi:10.1080/09544828.2016.1166482 · Cited by 185
Why certain systematic reviews reach uncertain conclusions
Petticrew, M. (2003) · BMJ · doi:10.1136/bmj.326.7392.756 · Cited by 181
Developing theory-driven design research
Cash, P. J. (2018) · Design Studies · doi:10.1016/j.destud.2018.03.002 · Cited by 176
A treatise on order in engineering design research
Horv�th, I. (2004) · Research in Engineering Design · doi:10.1007/s00163-004-0052-x · Cited by 170
The Medical Review Article Revisited: Has the Science Improved?
McAlister, F. A., Clark, H. D., van Walraven, C., Straus, S. E., Lawson, F. M. E., Moher, D., & Mulrow, C. D. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-131-12-199912210-00007 · Cited by 149
Hay, L., Duffy, A. H. B., McTeague, C. P., Pidgeon, L. M., Vuletic, T., & Grealy, M. (2017) · Design Science · doi:10.1017/dsj.2017.11 · Cited by 148
Validation of design methods: lessons from medicine
Frey, D., & Dym, C. L. (2006) · Research in Engineering Design · doi:10.1007/s00163-006-0016-4 · Cited by 146
Decision Making in Engineering Design
Lewis, K. E., Chen, W., Schmidt, L. C., & American Society of Mechanical Engineers (2006) · ASME Press eBooks · doi:10.1115/1.802469 · Cited by 117
Evidence‐based medicine and its role in ethical decision‐making
Borry, P., Schotsmans, P. T., & Dierickx, K. (2006) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2006.00548.x · Cited by 54
Reflections on diversity in design research
McMahon, C. A. (2012) · Journal of Engineering Design · doi:10.1080/09544828.2012.676634 · Cited by 49
Cited by
Komninos, A., Stefanis, V., & Garofalakis, J. (2023) · Multimodal Technologies and Interaction · doi:10.3390/mti7020022 · Cited by 16
Musarat, M. A., & Ručevskis, S. (2026) · Automation in Construction · doi:10.1016/j.autcon.2026.107130 · Cited by 1
Metadata
- Authors
- Lame, G.
- Venue
- Proceedings of the Design Society: International Conference on Engineering Design
- Publisher
- Cambridge University Press (CUP)
- Published
- 2019-07
- Volume
- 1
- Issue
- 1
- Pages
- 1633-1642
- ISSN
- 2220-4334 (ISSN-L)
- 2220-4342
- OpenAlex
- W3006244985
- Cited by
- 414
- References
- 46
- FWCI
- 2.38
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.