The DOI matches its Crossref record; 64 of the 67 references it cites were found in a registry or an index.
Experimental and quasi-experimental designs in implementation research
Open accessAbstract
Implementation science is focused on maximizing the adoption, appropriate use, and sustainability of effective clinical practices in real world clinical settings. Many implementation science questions can be feasibly answered by fully experimental designs, typically in the form of randomized controlled trials (RCTs). Implementation-focused RCTs, however, usually differ from traditional efficacy- or effectiveness-oriented RCTs on key parameters. Other implementation science questions are more suited to quasi-experimental designs, which are intended to estimate the effect of an intervention in the absence of randomization. These designs include pre-post designs with a non-equivalent control group, interrupted time series (ITS), and stepped wedges, the last of which require all participants to receive the intervention, but in a staggered fashion. In this article we review the use of experimental designs in implementation science, including recent methodological advances for implementation studies. We also review the use of quasi-experimental designs in implementation science, and discuss the strengths and weaknesses of these approaches. This article is therefore meant to be a practical guide for researchers who are interested in selecting the most appropriate study design to answer relevant implementation science questions, and thereby increase the rate at which effective clinical practices are adopted, spread, and sustained.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Quasiexperiments are appropriate for practical situations and ethical compliance for studies involving vulnerable participants such as older adults, 17 and allow scholars to conduct rigorous studies under certain limitations and non-control conditions. 18”
From Mixed reality technology for older adults: Evaluating the impact of a novel virtual humanoid coach in a community-based physical exercise program in the Philippines (2024) · cites this paper as [18]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721 · cited 1×
“Second, as a corollary of this point, implementation RCTs typically feature different outcome measures than efficacy or effectiveness RCTs, with an emphasis on the extent to which a health intervention was successfully implemented rather than an evaluation of the health effects of that intervention (Proctor et al., 20…”
BackgroundEvaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · cited 1×
“A variety of evaluation-oriented implementation frameworks may guide the choices of such measures (e.g., RE-AIM; Gaglio et al., 2013; Glasgow et al., 1999).”
BackgroundFisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012) · Medical Care · doi:10.1097/mlr.0b013e3182408812 · Cited by 4,058 · cited 1×
“Hybrid implementation-effectiveness studies attend to both effectiveness and implementation outcomes (Curran et al., 2012); these designs are also covered in more detail elsewhere in this issue (Landes, this issue).”
BackgroundSegmented regression analysis of interrupted time series studies in medication use research
Wagner, A. K., Soumerai, S. B., Zhang, F., & Ross-Degnan, D. (2002) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1046/j.1365-2710.2002.00430.x · Cited by 3,279 · cited 2×
“More formally, ITS evaluations focus on identifying whether there is discontinuity in the trend (change in slope or level) after the intervention relative to before the intervention, using segmented regression to model pre- and post-intervention trends (Gebski et al., 2012; Penfold and Zhang, 2013; Taljaard et al., 20…”
BackgroundInterrupted time series regression for the evaluation of public health interventions: a tutorial
Lopez Bernal, J., Cummins, S., & Gasparrini, A. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw098 · Cited by 3,049 · cited 4×
“Relative to an RCT, ITS designs can also allow for a more comprehensive assessment of the longitudinal effects of an intervention (positive or negative), as effects can be traced over all included time points (Bernal et al., 2017; Penfold and Zhang, 2013).”
BackgroundInfluentialImproving Primary Care for Patients With Chronic Illness
Bodenheimer, T. (2002) · JAMA · doi:10.1001/jama.288.14.1775 · Cited by 2,961
Organizing Care for Patients with Chronic Illness
Wagner, E. H., Austin, B. T., & Korff, M. V. (1996) · The Milbank Quarterly · doi:10.2307/3350391 · Cited by 2,877 · cited 1×
“, 2014) to implement care practices consistent with the collaborative chronic care model (CCM; Bodenheimer et al., 2002a, 2002b; Wagner et al., 1996) in nine outpatient mental health teams in VA medical centers.”
MethodTheory of Statistical Estimation
Fisher, R. A. (1925) · Mathematical Proceedings of the Cambridge Philosophical Society · doi:10.1017/s0305004100009580 · Cited by 2,684
Improving Primary Care for Patients With Chronic Illness: The Chronic Care Model, Part 2
Bodenheimer, T., Wagner, E. H., & Grumbach, K. (2002) · JAMA · doi:10.1001/jama.288.15.1909 · Cited by 2,413
Research works printed but not linked: 1
- Neumann, 2000, Replicating effective programs: HIV/AIDS prevention technology transfer, AIDS Educ. Prev., 12, 35
Cited by
Sorted by citations
Items 301–358 of 358
Parpio, Y. N., Nuruddin, R., Ali, T. S., Mohammed, N., Khan, U. R., Shahzad, S., Khan, M. M., & Aslam, M. (2025) · JMIR Research Protocols · doi:10.2196/67193
“The first section will gather detailed demographic information and include eligibility questions, followed by the Modified Scale for Suicidal Ideation (MSSI) by Miller et al [43].”
InfluentialEffectiveness of Cross-Age Tutoring on Problem-Solving Skills in Trigonometry Among Grade 9 Students
Guanga, J. A., Barañao, E., Lañojan, R., Bergante, J., & Andang-Alaiden, E. (2025) · Psychology and Education A Multidisciplinary Journal · doi:10.70838/pemj.491008
Ketharanathan, N., Figaji, A., Tibboel, D., & de Lange, E. D. (2025) · Intensive Care Medicine · doi:10.1007/s44253-025-00109-x
“Other trial approaches used in so-called implementation science include staggered approaches , which aim to assess the effect of an intervention in the absence of randomization [16–18].”
Interpreting QI Outcomes in Neonatal Infection Control: Methodology Matters.
Mathur, S., & Gupta, U. (2025) · Indian Pediatrics · doi:10.1007/s13312-025-00234-y
METODE PENYELIDIKAN DALAM PSIKOLOGI PERKEMBANGAN
Yulianti, Maulida, A., & Samsuddin (2025) · Al Irfan : Jurnal Ilmu Pendidikan dan Penelitian · doi:10.64877/alirfan.v1i2.41
“Penelitian jenis ini sangat umum dalam penelitian pendidikan, di mana administrator tidak mengizinkan pemilihan acak siswa untuk sampel eksperimental (Miller et al., 2020).”
The top 50 cited publications in the field of vitamin B12 deficiency: A bibliometric analysis review
Aldakhil, M. D., Aldakhil, H. A., Istanboli, A., Alamri, R. S., Aljuhani, L. I., Alahmed, M. A., Basalamah, K. A., Alzibali, Y. M., Almufleh, A. S., Harthi, O. M. A., et al. (2025) · Journal of Family Medicine and Primary Care · doi:10.4103/jfmpc.jfmpc_2096_24
Ponstein, K., Erkens, P. M. G., van Breukelen, G., Hamers, J. P. H., & Bleijlevens, M. H. C. (2025) · BMC Geriatrics · doi:10.1186/s12877-025-06508-1
“This type of design is more often recognized for their use in implementation studies where strict randomization may be infeasible or unfitting [61].”
Rui-hua, Z., Shi, K., Li, S., & Zhou, W. (2025) · PsyCh Journal · doi:10.1002/pchj.70054
“Quasi-experimental designs, while often more feasible in constrained settings such as the pandemic (Miller et al. 2020), may introduce bias.”
COGNITIVE MAPPING IN THE WRITING CLASSROOM
Putri, F., Abdullah, U., & Sukma, F. A. O. (2025) · Didascein : Journal of English Education · doi:10.52333/djoee.v6i2.1475
“Miller et al. (2020) stated that pre-post designs with a non-equivalent control group, interrupted time series (ITS), and stepped wedge designs were quasi-experimental.”
Kurnia, S., Montessori, M., Muhammadi, & Rahmi, U. (2025) · Jurnal Penelitian Pendidikan IPA · doi:10.29303/jppipa.v11i9.12361 · Cited by 2
“This statement is supported by the opinion that, in a Quasi-Experimental Design, the control group does not fully control the external variables that influence the experiment (Miller et al., 2020; Schwichow et al., 2016).”
Rocha, P. K., de Souza, I. D., Silva, T. L., de Moura, S. D., Vitor, M. Z., de Souza, S., & Anders, J. C. (2025) · Rev Rene · doi:10.36517/2175-6783.20252695534
Sovianti, V., Dwidiyanti, M., & Ismail, S. (2025) · JKG (Jurnal Keperawatan Global) · doi:10.37341/jkg.v10i1.724
“A purposive sample is the one whose characteristics are defined for a purpose that is relevant to the study (Miller et al., 2020).”
Patenting trends in biodegradable implant: An analysis
Sarkar, M., & Bandyopadhyay, T. K. (2025) · World Patent Information · doi:10.1016/j.wpi.2025.102378
Holm, K. G., Hägi‐Pedersen, M., Haslund‐Thomsen, H., Nilsson, I. M. S., Aagaard, H., Maastrup, R., Ejlertsen, C., Petersen, M., Feenstra, M. M., Kristensen, I. H., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-100258
Dupa, J., & BANTULO, J. S. (2025) · Psychology and Education A Multidisciplinary Journal · doi:10.70838/pemj.440102
Cevallos, P. D. R., & Padilla, L. D. N. (2025) · LATAM Revista Latinoamericana de Ciencias Sociales y Humanidades · doi:10.56712/latam.v6i3.4229
Sheraz, M., & Sajid, U. (2025) · Global Spine Journal · doi:10.1177/21925682251360743
The Effect of Religious Music Therapy on Reducing the Degree of Menstrual Pain in Adolescent Girls
Rahmat, R., Fitriani, D. D. D., Wulandari, P. P., Nurmiaty, Faisal, T., & Khaira, N. (2025) · Caring Jurnal Keperawatan · doi:10.29238/caring.v14i2.2784
“According to Miller, Smith, and Pugatch (2020), designs lacking control groups generally have weak internal validity because they cannot distinguish between actual treatment effects and naturally occurring changes.”
Effects of reflexology lymphatic drainage on lymphoedema among patients post mastectomy
Kamel, E., Sebaee, H. A., Hassan, M., & Deif, H. (2025) · Egyptian Nursing Journal · doi:10.4103/enj.enj_1_25
Novanti, T. S., Fitri, R., & Jannah, M. (2025) · Studies in Philosophy of Science and Education · doi:10.46627/sipose.v6i1.571
Lietzen, L., & Hjelholt, T. J. (2025) · The Lancet Healthy Longevity · doi:10.1016/j.lanhl.2025.100704
The edge of knowledge in systematic reviews and meta-analysis.
Mohammadi, M. (2025) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2025.104464
Big Talks for Little People: A Pilot Study of a Classroom Based Mental Health Program
Slee, P. T., Pill, S., & Agnew, D. (2025) · Health Promotion Journal of Australia · doi:10.1002/hpja.70014 · Cited by 2
Amani, O., Choolabi, R. H., & Peykani, S. (2025) · Current Psychology · doi:10.1007/s12144-025-07369-y
“In contrast, quasi-experimental designs, especially those with pretest-posttest structures and control groups, allow for more practical implementation, providing valuable insights in naturalistic environments, albeit with limitations in causal inference (Miller et al., 2020).”
Effects of socio-therapeutic interventions on cognitive functions among institutionalized elderly
Zhuang, Y., Botabara-Yap, M. J., Ramos, R. I. A., & Balila, J. S. (2024) · Experimental Gerontology · doi:10.1016/j.exger.2024.112629
Hashem, E. M., Saleh, A. A., Eloseily, A. M., Mohammed, D. F., & Ismael, M. A. E.-F. (2024) · Egyptian Journal of Health Care · doi:10.21608/ejhc.2024.379367
“In this design the investigator recruited two groups: one experimental and one control where the outcomes on the experimental group before and after the intervention were measured (Miller et al., 2020).”
MethodEffect of virtual reality on women’s labor pain perception at Tanta University Hospital
Abosaleh, E. M. A., Hameed, A. A., & Abdelmonem, A. S. (2024) · Egyptian Nursing Journal · doi:10.4103/enj.enj_25_24
KURILO, T., & Pentz, R. D. (2024) · Oncology Research Featuring Preclinical and Clinical Cancer Therapeutics · doi:10.32604/or.2024.050266
“These factors are key confounding factors as they in fl uence cancer survival outcomes [23,24].”
BackgroundArmiati, A., Nurhafizah, N., Hidayat, M., & Rahmidani, R. (2024) · Journal Of Education, Teaching and Learning · doi:10.26737/jetl.v9i1.5906
“More precisely in the form of a quasi-experiment (Almost experiment) because not all variables (symptoms that appear) and experimental conditions can be measured and controlled strictly (Madadizadeh, 2022; Miller et al,, 2020).”
BackgroundLevesque, A., & Roy-Wsiaki, G. N. (2024) · Health Education · doi:10.1108/he-08-2023-0090
Morgan, M. H. C., Herbst, J. H., Fortson, B. L., Shortt, J. W., Willis, L. A., Lokey, C. N., Slep, A. M. S., Lorber, M. F., & Huber‐Krum, S. (2024) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2024.106928 · Cited by 1
Pereira, C., Bambirra, E., Cruz, I. A., & Chemello, C. (2024) · Perspectivas em Diálogo: Revista de Educação e Sociedade · doi:10.55028/pdres.v11i27.20029
Eraso, C. J. G., Palomino, M. A. R., Sánchez, D. A. M., Ruiz, J. S. R., Cano, P. J. B., Pedraza, J. A. N., Vanegas, L. A. G., Fernández, L. M. M., Martínez, D., Ortega-Sierra, M. G., et al. (2024) · Horizonte Médico (Lima) · doi:10.24265/horizmed.2024.v24n2.05
“Asimismo, la experimentación podría explicarse como el resultado de los estudios observacionales a lo largo del tiempo, que han demostrado ciertas tendencias sobre el dolor crónico posoperatorio (22) .”
BackgroundNing, C., & Huang, Y. (2024) · Education and Information Technologies : Official Journal of the IFIP technical committee on Education · doi:10.1007/s10639-024-12824-7
“For some experimental studies that are not 1 3 Education and Information Technologies (2024) 29:25013–25034 easily conducted, a quasi-experimental research method can be used to design controls to minimize potential factors affecting the validity of the study (Miller et al., 2020).”
RetractedMohamed, M. K., Khalil, N. E. M., & Abdelaziz, F. S. (2024) · Egyptian Journal of Health Care · doi:10.21608/ejhc.2024.367110
“Or is a form of research where the investigator has no control over the independent variable but has power over how the dependent variable is measured according to medical dictionary definition (Miller, Smith, & Pugatch, 2019).”
MethodAfdarina, O., Chatri, M., Helendra, & Irdawati (2024) · Jurnal Penelitian Pendidikan IPA · doi:10.29303/jppipa.v10i5.7191
“In this design, control variables cannot be implemented strictly or fully (Miller et al., 2020).”
BackgroundIge, O., & Jita, T. (2024) · Ukrainian Journal of Educational Studies and Information Technology · doi:10.32919/uesit.2024.01.04
Amalia, D., Hanafi, I., & Taryudi (2024) · Journal of Scientific Research, Education, and Technology (JSRET) · doi:10.58526/jsret.v3i2.380
“According to (Miller et al., 2020), the following table displays the research design.”
BackgroundMethodimpact of the STAD model on motivating math learning in addition and subtraction
Tania, R., Pahmi, S., Hopeman, T. A., & Minasyan, S. (2024) · Union: Jurnal Ilmiah Pendidikan Matematika · doi:10.30738/union.v12i1.15936
Yudiatna, C. C. P., Kaswindiarti, S., Suparno, N. R., & Karyadi, E. (2024) · Proceeding ISETH (International Summit on Science, Technology, and Humanity) · doi:10.23917/iseth.4217
“This quantitative research uses a quasi-experimental type (quasi experiment), which has a control group, but its role cannot fully control other factors that influence the research (Millera, et al., 2020).”
Effect of monotonous 10-week TABATA workout for overweight female ages 19-21 years old
Ekowati, E., Habibie, H., Farinas, L. M., Inayan, L., Donguila, J., Pastrana, L., Ostan, F., Lera, D., Setiawan, E., Hofmeister, M., et al. (2023) · Fizjoterapia Polska · doi:10.56984/8zg20bd9c
Nugroho, D., Hidayatullah, M. F., Doewes, M., Purnama, S. K., Donguila, J., Inayan, F., Bedro, J., Setiawan, E., Lobo, J., & Németh, Z. (2023) · Fizjoterapia Polska · doi:10.56984/8zg20b8a7 · Cited by 1
Aragonès, E., Rodoreda, S., Guitart, M., Garcia, E., Berenguera, A., Martín, F., Rambla, C., Aragonès, G., Calvo, A., Mas, A., et al. (2023) · BMC Primary Care · doi:10.1186/s12875-023-02187-2
“However, the proposed methodology is completely appropriate for a implementation study [39, 40], where the objective is not so much to evaluate the effectiveness of the intervention, but rather focuses on concepts such as practicability in the context of real clinical practice, its feasibility, its…”
The Efficiency of Using Google Classroom to Teach Writing Descriptive Text
Saputri, K., Kuntarto, E., Sofyan, & Budiyanto, D. (2023) · Journal of English Education and Teaching · doi:10.33369/jeet.7.4.822-831
“Maciejewski (2020) and Miller et al. (2020).”
Syaripudin, A. (2023) · Jurnal Multidisiplin Madani · doi:10.55927/mudima.v3i9.6074 · Cited by 1
Noninferiority Clinical Trial of Adapted START NOW Psychotherapy for Outpatient Opioid TreatmentPreprint
Truong, A. Y.-Q., Kablinger, A. S., Hartman, C. W., Hartman, D., West, J., Hanlon, A., Lozano, A. J., McNamara, R., Seidel, R. W., & Trestman, R. L. (2023) · Research Square · doi:10.21203/rs.3.rs-3229052/v1
Race-Related Disparities in Childhood Adversity, Trauma, and Stress-Related Psychopathology.
Kalin, N. (2023) · American Journal of Psychiatry · doi:10.1176/appi.ajp.20221026
Opozda, M. J., Oxlad, M., Turnbull, D. A., Gupta, H., Vincent, A. D., Ziesing, S., Nankivell, M. E., & Wittert, G. (2022) · Journal of Medical Internet Research · doi:10.2196/40854
Sayed, R., & SadekRamadanFayed, S. (2021) · Recommandations pour un transport durable respectueux de l’environnement et de la santé - « Reconstruire en mieux » · doi:10.18356/9789210056922c002
“In addition to the pre-test and immediate post-test, a hindered post-test or post-tests are often encompassed to survey the treatment effects over the longer term (Miller et al., 2020).”
Les études interventionnelles ou évaluatives
Jacq, G., & Legriel, S. (2020) · La Revue de l Infirmière · doi:10.1016/s1293-8505(20)30244-x
Les différents schémas d’études épidémiologiques et leurs applications
Jacq, G., & Legriel, S. (2019) · La Revue de l Infirmière · doi:10.1016/j.revinf.2019.10.017
Cabrera-Duffaut, A., Pinto-Llorente, A. M., & Iglesias-Rodríguez, A. (2026) · Journal of Interactive Media in Education · doi:10.5334/jime.1029
Camarillo, M. (2026) · Journal of interdisciplinary perspectives · doi:10.69569/jip.2026.323
Prasetya, F., Ayu, R., Fajri, B. R., Mulianti, M., Hamdani, H., Indrawan, E., Parezi, M. I., Raihan, M., & Fortuna, A. (2026) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2026.103580
Eñate, P. (2026) · Journal of Interdisciplinary Perspectives · doi:10.69569/jip.2026.429x
Okubo, C. V. C., Martins, J. T., da Silva, M. M. T., Galdino, M. J., Haddad, M. D. C. F. L., Cardelli, A. A. M., & Silveira, R. C. D. C. P. (2022) · Revista Latino-Americana de Enfermagem · doi:10.1590/1518-8345.5923.3640
Implementation Science: Developing a Common Understanding for Use in Mental Health
Gutner, C. A., & Wells, S. Y. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-818697-8.00086-8
Dissemination and Implementation Research
Barnett, M. L., Dickson, K. S., Becker‐Haimes, E. M., & Stadnick, N. A. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-818697-8.00170-9 · Cited by 4
Metadata
- Authors
- Miller, C. J., Smith, S. N., & Pugatch, M.
- Affiliations
- United States Department of Veterans Affairs (US)
- University of Michigan (US)
- Corresponding author
- Miller, Christopher J.
- Venue
- Psychiatry Research
- Publisher
- Elsevier BV
- Published
- 2020-01
- Volume
- 283
- Pages
- 112452
- ISSN
- 0165-1781 (ISSN-L)
- PMC
- PMC6923620
- OpenAlex
- W2951715721
- Cited by
- 373
- References
- 67
- Citations by year
- 2026: 63 · 2025: 89 · 2024: 67 · 2023: 63 · 2022: 56 · 2021: 20 · 2020: 14
- FWCI
- 4.18
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Psychiatry Research (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health
- U.S. Department of Veterans Affairs
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- implementation research, implementation science, experimental design, quasi-experimental design, randomized controlled trials, interrupted time series, stepped wedge design, pre-post design, research design selection, clinical adoption, intervention sustainability
- MeSH terms
- Implementation Science, Humans, Research Design, Randomized Controlled Trials as Topic, Control Groups, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
