The DOI matches its Crossref record; 64 of the 67 references it cites were found in a registry or an index.
Experimental and quasi-experimental designs in implementation research
Open accessAbstract
Implementation science is focused on maximizing the adoption, appropriate use, and sustainability of effective clinical practices in real world clinical settings. Many implementation science questions can be feasibly answered by fully experimental designs, typically in the form of randomized controlled trials (RCTs). Implementation-focused RCTs, however, usually differ from traditional efficacy- or effectiveness-oriented RCTs on key parameters. Other implementation science questions are more suited to quasi-experimental designs, which are intended to estimate the effect of an intervention in the absence of randomization. These designs include pre-post designs with a non-equivalent control group, interrupted time series (ITS), and stepped wedges, the last of which require all participants to receive the intervention, but in a staggered fashion. In this article we review the use of experimental designs in implementation science, including recent methodological advances for implementation studies. We also review the use of quasi-experimental designs in implementation science, and discuss the strengths and weaknesses of these approaches. This article is therefore meant to be a practical guide for researchers who are interested in selecting the most appropriate study design to answer relevant implementation science questions, and thereby increase the rate at which effective clinical practices are adopted, spread, and sustained.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Quasiexperiments are appropriate for practical situations and ethical compliance for studies involving vulnerable participants such as older adults, 17 and allow scholars to conduct rigorous studies under certain limitations and non-control conditions. 18”
From Mixed reality technology for older adults: Evaluating the impact of a novel virtual humanoid coach in a community-based physical exercise program in the Philippines (2024) · cites this paper as [18]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-010-0319-7 · Cited by 8,721 · cited 1×
“Second, as a corollary of this point, implementation RCTs typically feature different outcome measures than efficacy or effectiveness RCTs, with an emphasis on the extent to which a health intervention was successfully implemented rather than an evaluation of the health effects of that intervention (Proctor et al., 20…”
BackgroundEvaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · cited 1×
“A variety of evaluation-oriented implementation frameworks may guide the choices of such measures (e.g., RE-AIM; Gaglio et al., 2013; Glasgow et al., 1999).”
BackgroundFisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012) · Medical Care · doi:10.1097/mlr.0b013e3182408812 · Cited by 4,058 · cited 1×
“Hybrid implementation-effectiveness studies attend to both effectiveness and implementation outcomes (Curran et al., 2012); these designs are also covered in more detail elsewhere in this issue (Landes, this issue).”
BackgroundSegmented regression analysis of interrupted time series studies in medication use research
Wagner, A. K., Soumerai, S. B., Zhang, F., & Ross-Degnan, D. (2002) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1046/j.1365-2710.2002.00430.x · Cited by 3,279 · cited 2×
“More formally, ITS evaluations focus on identifying whether there is discontinuity in the trend (change in slope or level) after the intervention relative to before the intervention, using segmented regression to model pre- and post-intervention trends (Gebski et al., 2012; Penfold and Zhang, 2013; Taljaard et al., 20…”
BackgroundInterrupted time series regression for the evaluation of public health interventions: a tutorial
Lopez Bernal, J., Cummins, S., & Gasparrini, A. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw098 · Cited by 3,049 · cited 4×
“Relative to an RCT, ITS designs can also allow for a more comprehensive assessment of the longitudinal effects of an intervention (positive or negative), as effects can be traced over all included time points (Bernal et al., 2017; Penfold and Zhang, 2013).”
BackgroundInfluentialImproving Primary Care for Patients With Chronic Illness
Bodenheimer, T. (2002) · JAMA · doi:10.1001/jama.288.14.1775 · Cited by 2,961
Organizing Care for Patients with Chronic Illness
Wagner, E. H., Austin, B. T., & Korff, M. V. (1996) · The Milbank Quarterly · doi:10.2307/3350391 · Cited by 2,877 · cited 1×
“, 2014) to implement care practices consistent with the collaborative chronic care model (CCM; Bodenheimer et al., 2002a, 2002b; Wagner et al., 1996) in nine outpatient mental health teams in VA medical centers.”
MethodTheory of Statistical Estimation
Fisher, R. A. (1925) · Mathematical Proceedings of the Cambridge Philosophical Society · doi:10.1017/s0305004100009580 · Cited by 2,684
Improving Primary Care for Patients With Chronic Illness: The Chronic Care Model, Part 2
Bodenheimer, T., Wagner, E. H., & Grumbach, K. (2002) · JAMA · doi:10.1001/jama.288.15.1909 · Cited by 2,413
Research works printed but not linked: 1
- Neumann, 2000, Replicating effective programs: HIV/AIDS prevention technology transfer, AIDS Educ. Prev., 12, 35
Cited by
Sorted by citations
Designing and undertaking randomised implementation trials: guide for researchers
Wolfenden, L., Foy, R., Presseau, J., Grimshaw, J., Ivers, N. M., Powell, B. J., Taljaard, M., Wiggers, J., Sutherland, R., Nathan, N., et al. (2021) · BMJ · doi:10.1136/bmj.m3721 · Cited by 287
Reid, R. J., Wodchis, W. P., Kuluski, K., Lee-Foon, N. K., Lavis, J. N., Rosella, L. C., & Desveaux, L. (2024) · SSM - Health Systems · doi:10.1016/j.ssmhs.2024.100010 · Cited by 109
Designs and methods for implementation research: Advancing the mission of the CTSA program
Hwang, S., Birken, S. A., Melvin, C. L., Rohweder, C. L., & Smith, J. D. (2020) · Journal of Clinical and Translational Science · doi:10.1017/cts.2020.16 · Cited by 99
Thomas, D. R., Lin, J., Gatz, E., Gurung, A., Gupta, S., Norberg, K., Fancsali, S. E., Aleven, V. A. W. M. M., Branstetter, L., Brunskill, E., et al. (2024) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3636555.3636896 · Cited by 91
“Randomized controlled trials (RCTs), are universally accepted as the ideal methodology for assessing intervention efficacy and impact [23].”
BackgroundMethodInfluentialA Meta-analysis of University STEM Summer Bridge Program Effectiveness
Bradford, B. C., Beier, M. E., & Oswald, F. L. (2021) · CBE—Life Sciences Education · doi:10.1187/cbe.20-03-0046 · Cited by 73
Clark, E., Cranston, E., Polin, T., Ndumbe-Eyoh, S., MacDonald, D., Betker, C., & Dobbins, M. (2022) · BMC Public Health · doi:10.1186/s12889-022-14603-w · Cited by 88
Cabrera-Solano, P. (2022) · International Journal of Educational Methodology · doi:10.12973/ijem.8.4.719 · Cited by 71
“This type of design involves the application of pre and post-tests, and allows the researchers to conduct rigorous studies (Miller et al., 2020).”
BackgroundEvidence-based practice implementation in healthcare in China: a living scoping review
Zhao, J., Bai, W., Zhang, Q., Su, Y., Wang, J., Xiao-ning, D., Zhou, Y., Kong, C., Qing, Y., Gong, S., et al. (2022) · The Lancet Regional Health - Western Pacific · doi:10.1016/j.lanwpc.2021.100355 · Cited by 63
Otache, I., Edopkolor, J. E., & Kadiri, U. (2022) · The International Journal of Management Education · doi:10.1016/j.ijme.2022.100645 · Cited by 56
Hsia, L., Lin, Y., Lin, C.-H., & Hwang, G. (2024) · Computers & Education · doi:10.1016/j.compedu.2024.105212 · Cited by 57
CRISPR‐driven diagnostics: Molecular mechanisms, clinical efficacy and translational challenges
Wang, Z., Wang, Q., Zhang, J., Li, B., Li, Y., Chen, Z., Guo, D., & Feng, S. (2025) · Clinical and Translational Medicine · doi:10.1002/ctm2.70482 · Cited by 51
“132,133”
Zhang, W., & Li, J. (2023) · Urban forestry & urban greening · doi:10.1016/j.ufug.2023.127898 · Cited by 49
“Another advantage of ITS design is that it involves a before-and-after comparison within a single group instead of a comparison with a control group, which is appropriate for research on COVID-19, because no control group was available since the outbreak occurred on a worldwide scale (Bernal et al., 2018b; Miller et a…”
BackgroundTikTok for developing learning motivation and oral proficiency in MICE learners
Gao, S.-Y., Tsai, Y.-Y., Huang, J.-H., Ma, Y.-X., & Wu, T.-L. (2023) · Journal of Hospitality Leisure Sport & Tourism Education · doi:10.1016/j.jhlste.2022.100415 · Cited by 52
Donohue, J. F., Elborn, J. S., Lansberg, P. J., Javed, A., Tesfaye, S. H., Rugo, H. S., Devi, R., Jithoo, N., Huang, P.-H., Subramaniam, K., et al. (2023) · Journal of Healthcare Leadership · doi:10.2147/jhl.s394088 · Cited by 56
“86 ● Refinement of clinical practice guidelines based on RCTs taking into consideration emerging data from real-world clinical practice.”
A typology of quantitative approaches to discovery
Dolničar, S., Zinn, A. K., & Demeter, C. (2023) · Annals of Tourism Research · doi:10.1016/j.annals.2023.103704 · Cited by 41
Hu, Y., Zhang, S., Shi, Y., & Guo, L. (2023) · Scientific Reports · doi:10.1038/s41598-023-29937-7 · Cited by 39
“(7) Q Sup s,t1 = − ∑ k∈KESV Naive s,k,t1 − ∑ k∈KESV Naive s,k,t0”
Griffiths, M., Creedy, D. K., Carter, A. G., & Donnellan-Fernandez, R. E. (2022) · Nurse Education in Practice · doi:10.1016/j.nepr.2022.103349 · Cited by 38
Advancing climate change health adaptation through implementation science
Neta, G. I., Pan, W., Ebi, K. L., Buss, D. F., Castranio, T., Lowe, R., Ryan, S. J., Ibarra, A. M. S., Hapairai, L. K., Sehgal, M., et al. (2022) · The Lancet Planetary Health · doi:10.1016/s2542-5196(22)00199-1 · Cited by 53
Evaluating technology integration in education: a framework for professional development
Siyam, Y., Siyam, N., Hussain, M., & Alqaryouti, O. (2025) · Discover Education · doi:10.1007/s44217-025-00448-z · Cited by 34
Zou, C., Harvard, A., Qian, J., & Fox, B. I. (2023) · DIGITAL HEALTH · doi:10.1177/20552076231191050 · Cited by 34
“RCTs, quasi-experimental studies, and qualitative studies) yield different results, with RCTs generally considered more robust than quasi-experimental studies.(78) Also, the inclusion of diverse populations could lead to various experiences of DHTs.”
ResultQi, Y. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2301256 · Cited by 36
“However, they can still reveal the relationships and impact of the studied factors, providing valuable information and support (Miller et al., 2020).”
Wang, C., Zhu, Y., Ma, J., & Chu, J. (2023) · Digital Health · doi:10.1177/20552076231207587 · Cited by 33
“48,49 Other approaches also have their limitations, such as FE can only eliminate time-fi xed confounding terms 50 and PSM that rely exclusively on the hypothesis that “ observables are ignored.”
BackgroundResultImplementation Science Methodologies for Complementary and Integrative Health Research
Clark, D. M., Edwards, E., Murray, P., & Langevin, H. M. (2021) · The Journal of Alternative and Complementary Medicine · doi:10.1089/acm.2020.0446 · Cited by 23
“Cluster randomized designs, in which sites or clinics rather than individual patients are the units of randomization, are common.(40,41) When combinations of implementation strategies are being assessed, factorial or fractional-factorial designs that randomize sites or providers to different combinations of strategies…”
BackgroundMethodResultInfluentialZuo, R., Li, W., & Zhang, X. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1540169 · Cited by 17
“In experimental designs, participants were randomly assigned to either the treatment group or the control group, whereas in the quasi-experimental studies, participants were not randomly assigned (Miller et al., 2020).”
Mohanna, Z., Kusljic, S., & Jarden, R. (2021) · Australian Critical Care · doi:10.1016/j.aucc.2021.05.012 · Cited by 23
Park, J., Lee, H., Kim, Y., Norton, C., Woodward, S., & Lee, S. (2023) · International Neurourology Journal · doi:10.5213/inj.2346014.007 · Cited by 20
“A QES is an experimental study without randomization or a control group [23].”
MethodMethodological Aspects in Randomized Clinical Trials of Nutritional Interventions
Martínez‐López, E., Pérez-Guerrero, E. E., Torres-Carrillo, N. M., López-Quintero, A., Betancourt‐Núñez, A., & Gutiérrez-Hurtado, I. A. (2022) · Nutrients · doi:10.3390/nu14122365 · Cited by 24
“A RCT, regardless of the type of intervention, is essentially a research design that must meet two requirements: the manipulation of a variable (the independent variable), and the random allocation of the intervention between study groups [6].”
BackgroundLi, J., Zhou, X., & Wang, Q. (2022) · Applied Psychology Health and Well-Being · doi:10.1111/aphw.12375 · Cited by 27
Fu, C., Chien, W. T., Zhang, Y., & Lam, S. K. K. (2025) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-025-02741-6 · Cited by 23
“…15 [62, 66, 70–72, 74, 76, 78, 81, 89, 91, 96, 98, 102, 104] used an quasi-experimental design, which refers to non-randomized pre–post-tests with a control group [108], two [88, 106] used quasi-c-RCTs, which refers to the absence of randomization, and one [105] used a Solomon four-group design.”
Building a Life Course Intervention Research Framework
Russ, S. A., Hotez, E., Berghaus, M., Hoover, C. G., Verbiest, S., Schor, E. L., & Halfon, N. (2022) · PEDIATRICS · doi:10.1542/peds.2021-053509e · Cited by 22
Rosadiño, J. D. T., Pagtakhan, R. G., Briñes, M. T., Dinglasan, J. L. G., Cruz, D. P., Corciega, J. O. L., Pagtakhan, A. B., Regencia, Z. J. G., & Baja, E. S. (2021) · medRxiv · doi:10.1371/journal.pone.0282644 · Cited by 18
“This community-based, volunteer-run organization offers HIV and other sexually-health-related services in Metro Manila, Philippines.”
InfluentialIbrahim, N. N., Ayub, A., & Yunus, A. S. (2020) · International Journal of Learning, Teaching and Educational Research · doi:10.26803/ijlter.19.7.14 · Cited by 18
Zgambo, M., Anyango, E., Arabiat, D. H., Ngune, I., Mörelius, E., Zhang, M., & Whitehead, L. (2025) · JMIR Pediatrics and Parenting · doi:10.2196/70745 · Cited by 15
“The larger effect observed in the quasi-experiment could be because of the lack of randomization in these studies [59].”
Zhang, S., & Ma, C. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-062272 · Cited by 19
Designing clinically useful psychopharmacological trials: challenges and ways forward
Chevance, A., Ravaud, P., Cornelius, V. R., Mayo‐Wilson, E., & Furukawa, T. A. (2022) · The Lancet Psychiatry · doi:10.1016/s2215-0366(22)00041-4 · Cited by 18
Jayadiningrat, M. G., Widiana, I. W., Ariani, N. W. T., & Widiani, N. K. (2022) · Jurnal Pedagogi dan Pembelajaran · doi:10.23887/jp2.v5i3.50268 · Cited by 20
The ‘Tasty School’ model is feasible for food education in primary schools
Laitinen, A. L., Antikainen, A., Mikkonen, S. J., Kähkönen, K., Talvia, S. S., Varjonen, S., Paavola, S., Karhunen, L. J., & Tilles‐Tirkkonen, T. (2022) · Journal of Human Nutrition and Dietetics · doi:10.1111/jhn.13071 · Cited by 26
Kartimi, K., & Winarso, W. (2021) · Al Ibtida: Jurnal Pendidikan Guru MI · doi:10.24235/al.ibtida.snj.v8i2.9175 · Cited by 14
“However, it did not fully function to control for external variables that affect the execution of the experiment (Miller, Smith & Pugatch, 2020).”
Abdullah, A. H., Misrom, N. S. B., Kohar, U. A., Hamzah, M. H., Ashari, Z. M., Ali, D. F. B., Samah, N., Tahir, L., & Rahman, S. N. S. A. (2020) · IEEE Access · doi:10.1109/access.2020.3014202 · Cited by 14
“It appears that adequate time is essential to ensure that a given treatment in an experimental group is effective [76].”
BackgroundFebrianti, R., Yufrizal, A., Putra, R. P., & Phongdala, P. (2023) · Jurnal Pendidikan Teknologi Kejuruan · doi:10.24036/jptk.v6i4.34523 · Cited by 13
“The design used in this study is to use an equivalent control group design (Miller et al., 2020).”
MethodLaitinen, A. L., Antikainen, A., Mikkonen, S. J., Kähkönen, K., Talvia, S. S., Varjonen, S., Paavola, S., Karhunen, L. J., & Tilles‐Tirkkonen, T. (2022) · Public Health Nutrition · doi:10.1017/s1368980022002154 · Cited by 20
“Setting and participants The present study had a quasi-experimental design(30) with fifteen intervention schools, which were supported to implement food education based on a tailored, teacherdelivered Tasty School model and ten control schools that”
MethodImplementation Science to Improve Quality of Neurological Care
Samanta, D., & Landes, S. J. (2021) · Pediatric Neurology · doi:10.1016/j.pediatrneurol.2021.05.009 · Cited by 16
Organic food research: Key contributors, research hotspots, and emerging trends
Azizan, A., Awal, A., Zain, W. Z. W. M., Hairoman, N., Endrini, S., & Hassan, F. (2025) · Applied Food Research · doi:10.1016/j.afres.2025.101109 · Cited by 10
Clark, E., Dhaliwal, B., Ciliska, D., Neil-Sztramko, S., Steinberg, M., & Dobbins, M. (2021) · Implementation Science Communications · doi:10.1186/s43058-022-00267-5 · Cited by 12
“In future iterations, a time series or stepped wedge design, where organizations are observed prior to intervention, may help further determine the program’s specific effects [53].”
BackgroundZhu, Y., Lu, Y., Xie, H. X., Ye, J., & Chen, M. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104365 · Cited by 11
THE EFFECTS OF SOCIO-SCIENTIFIC INQUIRY BASED LEARNING ON STUDENTS' PROBLEM-SOLVING SKILLS
Nugroho, A. A., Sajidan, S., Suranto, S., & Masykuri, M. (2025) · Journal of Baltic Science Education · doi:10.33225/jbse/25.24.149 · Cited by 11
Li, J., Xu, Z., Zhou, H. C., Li, Z., & Yuan, B. (2025) · BMC Family Practice · doi:10.1186/s12875-025-02743-y · Cited by 15
“This may be attributed to the well-controlled design of RCTs, which could get a more reliable estimation of effects [52].”
Kasa, A. S., Traynor, V., & Drury, P. (2024) · BMC Geriatrics · doi:10.1186/s12877-024-04909-2 · Cited by 11
“The study was conducted using a pre-, post-, and follow-up single group quasi-experimental design [36–38].”
MethodScholten, S., Schemer, L., Herzog, P., Haas, J. W., Heider, J., Winter, D., Reis, D., & Glombiewski, J. A. (2024) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-024-01363-5 · Cited by 19
“Therefore, pre-and post-implementation need to be clearly distinguished (Miller et al., 2020).”
Shen, L., Jiang, S., & Tan, S. (2024) · Research on Social Work Practice · doi:10.1177/10497315241227147 · Cited by 10
“The RCT design, often dubbed the “ gold standard ” for interventions, entails random allocation of a representative population sample into experimental and control groups, followed by intervention application (Miller et al., 2020).”
Carbonero, M. Á., Arteaga-Cedeño, W. L., Antón, L. J. M., & Molinero-González, P. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192315702 · Cited by 29
“We adopted an experimental pre-test and post-test research design with a control group [40].”
MethodOkubo, C. V. C., Martins, J., da Silva, M. M. T., Galdino, M. J., Haddad, M., Cardelli, A. A. M., & Silveira, R. C. D. C. P. (2022) · Revista Latino-Americana de Enfermagem · doi:10.1590/1518-8345.5923.3639 · Cited by 11
“In addition to that, effectiveness can also be affected by the follow-up time of the studies, as research has found that the outcomes can improve with increased study follow-up times(22).”
BackgroundInfluentialSeow, H. Y., Wu, M. H. L., Mohan, M., Mamat, N. H., Kutzsche, H. E., & Pau, A. K. H. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03455-y · Cited by 14
Alsaqer, K., & Alhmoud, S. H. (2025) · International Emergency Nursing · doi:10.1016/j.ienj.2025.101662 · Cited by 13
Han, Y., Han, Y., Huang, W., Liu, Y., Wang, Z., Zhao, W. K., & Zhang, W. (2025) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2025.105100 · Cited by 9
Harnessing Implementation Science in Clinical Psychology: Past, Present, and Future
Beidas, R. S., Boyd, M. R., Casline, E. P., Scott, K., Patel, Z. S., Mills, C., Mustanski, B. S., Schriger, S. H., Williams, F. S., Waller, C., et al. (2025) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-081423-021727 · Cited by 13
“The Implementation Research Logic Model (IRLM) (Figure 1a) is a blueprint for integrating multiple FTMs that specifies the relationships between core IS components (Smith et al. 2020).”
Emaliyawati, E., Ibrahim, K., Trisyani, Y., & Songwathana, P. (2025) · Advances in Medical Education and Practice · doi:10.2147/amep.s489163 · Cited by 10
“20 This research is part of the first author’s dissertation and is a trial of the learning model at stage III.”
Becker, A. M. (2023) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2023.1190412 · Cited by 9
THE EFFECTS OF SCENARIOS ON DECISION-MAKING QUALITY IN EARLY DESIGN – AN EMPIRICAL STUDY
Parolin, G., McAloone, T., & Pigosso, D. (2023) · Proceedings of the Design Society · doi:10.1017/pds.2023.338 · Cited by 10
Robles-Sanchez, M.-A., Bujan, P. A., Bosch‐Farré, C., Coll, C., Arévalo, M. J., Anglada, E., Menéndez, R., Montalbán, X., Sastre‐Garriga, J., Ramió‐Torrentà, L., et al. (2023) · Frontiers in Neurology · doi:10.3389/fneur.2023.1172640 · Cited by 11
“The common goal of this kind of study is to find whether a health intervention affects the participants and their clinical variables and health-related outcomes (13).”
BackgroundLi, J., Sun, X., & Wu, X. (2022) · Frontiers in Nutrition · doi:10.3389/fnut.2022.1009666 · Cited by 10
Lan, T., Wang, H., Li, X., Yin, H., Shao, D., Jiang, Y.-Y., & Yu, Q. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08320-8 · Cited by 10
Go, M. B., Golbin, R. A., Velos, S. P., Dayupay, J. P., Dionaldo, W., Cababat, F. G., Bongo, M. F., Troussas, C., & Ocampo, L. A. (2022) · International Journal of Mathematical Education in Science and Technology · doi:10.1080/0020739x.2022.2089604 · Cited by 23
“…unique to implementation science, can satisfactorily serve their purpose, especially in cases when experimental designs would become inappropriate (Miller et al., 2020) due to, for instance, in the case of this work, the difference of programmes, topics, and assigned instructors for each course.”
BackgroundGuyette, E. J., & Harris, S. M. (2024) · Family Court Review · doi:10.1111/fcre.12797 · Cited by 9
Reis, N., Dias, M. J. C., Sousa, L., Canedo, F., Rico, M., Henriques, M. A., & Baixinho, C. (2023) · Healthcare · doi:10.3390/healthcare11182561 · Cited by 9
“…pilot pre-post with a non-equivalent control group study was justified because the major goals of this type of study are to assess the feasibility and to avoid the potentially disastrous consequences of embarking on a large study, which could potentially “drown” the whole research effort [21].”
BackgroundChing, H., Chua, J. Y. X., Chua, J. S., & Shorey, S. (2023) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.12673 · Cited by 13
Implementation Science: Increasing the Public Health Impact of Audiology Research
Studts, C. R. (2022) · American Journal of Audiology · doi:10.1044/2022_aja-21-00205 · Cited by 14
“…research, for others, rigorous quasi-experimental designs are better suited (e.g., pre–post designs with nonequivalent control groups, interrupted time series, stepped wedge, sequential multiple assignment trials [SMART], and others; Hull et al., 2019; Miller et al., 2020; Schliep et al., 2017).”
Meta-Analyses Do Not Establish Improved Mortality With Ivermectin Use in COVID-19
Rothrock, S. G., Weber, K. D., Giordano, P. A., & Barneck, M. D. (2021) · American Journal of Therapeutics · doi:10.1097/mjt.0000000000001461 · Cited by 13
Su, L., Chen, H., & Lin, Z. (2025) · Tourism Management · doi:10.1016/j.tourman.2025.105228 · Cited by 12
Multicomponent Deimplementation Strategy to Reduce Low-Value Preoperative Testing.
Antunez, A. G., Kazemi, R. J., Richburg, C. E., Pesavento, C. M., Vastardis, A. F., Kim, E., Kappelman, A. L., Nanua, D., Pediyakkal, H., Jacobson-Davies, F. E., et al. (2025) · JAMA Surgery · doi:10.1001/jamasurg.2024.6063 · Cited by 8
Self-citationBernardinelli, F. C. P., Nascimento, J. D. S. G., do Nascimento, K. G., de Amorim, G. C., da Silva, S. A., & Chaváglia, S. R. R. (2022) · Revista Enfermagem UERJ · doi:10.12957/reuerj.2022.67137 · Cited by 6
Lamers‐Johnson, E., Kelley, K., Knippen, K. L., Feddersen, K., Sánchez, D. M., Parrott, J. S., Colin, C. R., Papoutsakis, C., & Jimenez, E. Y. (2022) · Frontiers in Nutrition · doi:10.3389/fnut.2022.969360 · Cited by 13
“This quasi-experimental study used a controlled pre-post design and was conducted from May 2017 to June 2019 (see Supplementary material for study timeline) (12, 13).”
BackgroundMethodGe, S., Leng, C.-H., & Shaharom, M. S. N. (2022) · International Journal of Chinese Education · doi:10.1177/2212585x221144901 · Cited by 8
“A quasi-experimental research design is for demonstrating the ef fi cacy of interventions, which is a popular and rigorous experimental methodology (Miller et al., 2020).”
MethodLumbantoruan, J. H. (2022) · Utamax Journal of Ultimate Research and Trends in Education · doi:10.31849/utamax.v4i2.9918 · Cited by 6
“This type of research is quantitative a QuasiExperimental (Miller et al., 2020).”
The Use of Augmented Reality in History Education: A Study on Conceptual Understanding Effects
Priyono, C., Sok, V., & Souza, F. (2025) · Journal Neosantara Hybrid Learning · doi:10.70177/jnhl.v2i3.1613 · Cited by 7
“To ensure a diverse and representative sample, 120 students are selected through stratified random sampling (Miller, 2020).”
Hyzak, K. A., Riccardi, J. S., Kinney, A. R., Esterov, D., Bogdanova, Y., & Bogner, J. A. (2024) · Journal of Head Trauma Rehabilitation · doi:10.1097/htr.0000000000000990 · Cited by 9
Chopra, R. C., Chakrabarthi, S., Narayan, I., & Chakraborty, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-75262-y · Cited by 8
“To evaluate the suitability of our setup for a natural experiment, we note that in line with traditional randomized control trial (RCT) setups, our sample is cluster-randomized at the site (apartment) level 36 .”
Macedo‐Rouet, M., Saux, G., Potocki, A., Dujardin, É., Dyoniziak, Y., Pylouster, J., & Rouet, J. (2024) · Instructional Science · doi:10.1007/s11251-024-09676-6 · Cited by 6
“We used a quasi-experimental pre-post design with a non-equivalent control group (Miller et al., 2020).”
Nasution, R., K., A., & Fernanda, Y. (2024) · Journal of Engineering Researcher and Lecturer · doi:10.58712/jerel.v3i2.127 · Cited by 7
“Using an experimental class and a control class, this method can identify differences in results between the treatment class and the control class (Miller et al., 2020).”
MethodYie, D. L., Sanmugam, M., Yahaya, W., & Khlaif, Z. N. (2024) · Higher Education for the Future · doi:10.1177/23476311241248994 · Cited by 7
“The approach involved participants selected into one of several distinct treatment groups, which ideally should be similar to compare the effectiveness of the interventions before and after the experiment (Maciejewski, 2020; Miller et al., 2020).”
Brausch-Böger, M. E., & Förster, M. (2024) · Education Sciences · doi:10.3390/educsci14050485 · Cited by 13
Mutual Aid Groups for Loneliness, Psychosocial Disability, and Continuity of Care
Agudelo‐Hernández, F., Guapacha-Montoya, M., & Andrade, R. M. R. (2024) · Community Mental Health Journal · doi:10.1007/s10597-023-01216-9 · Cited by 6
“In addition, collecting a sufficient number of data points between interruptions is crucial to ensure that the effects of each intervention can be determined individually (Miller et al., 2020).”
Intervention Effect of New Tuberculosis Control Model on Tuberculosis Incidence in Xinjiang
Zhang, Y., Wang, X.-Q., Liu, N.-Q., Wang, S.-L., Wang, X.-M., & Cao, M.-Q. (2023) · Infection and Drug Resistance · doi:10.2147/idr.s441899 · Cited by 7
“5 In the real world, policy evaluation usually relies on evidence from natural experiments or observational studies, and quasi-experimental design can be used to identify the real causal relationship between policy variables and target variables.”
BackgroundElevating Social Sciences Learning Outcomes: TGT Type Cooperative Learning Model
Pada, A., & Amir, F. (2022) · Jurnal Ilmiah Sekolah Dasar · doi:10.23887/jisd.v6i4.54046 · Cited by 7
Hughes, P. M., McGrath, R., & Thomas, K. (2022) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2022.12.006 · Cited by 7
Dykes, P., Lowenthal, G., Faris, A., Leonard, M., Hack, R. R., Harding, D., Whnp, C. H., Hurley, A., & An, P. G. (2020) · Journal of patient safety · doi:10.1097/pts.0000000000000790 · Cited by 7
An Evaluation of the Implementation of a UK School-Based Running Program
Chalkley, A., Routen, A., Harris, J. P., Cale, L. A., Gorely, T., & Sherar, L. (2020) · Children · doi:10.3390/children7100151 · Cited by 7
“Furthermore, it was believed to yield naturally occurring differences in types or intensities of implementation strategies, as determined by the schools and their local contexts, in order to evaluate their influence on implementation [19].”
BackgroundOlowo, S., Luwaga, R., & Niyonzima, V. (2025) · Open Access Emergency Medicine · doi:10.2147/oaem.s521517 · Cited by 4
Krismanuel, H., & Tjhin, P. (2025) · PLoS ONE · doi:10.1371/journal.pone.0325653 · Cited by 5
“Because the significance value is 0.309 > 0.05, it is possible to draw the conclusion that there is no meaningful relationship between the pre-test and post-test variables [24–27].”
InfluentialGonzales, J., & Estremera, M. L. (2025) · International Research and Innovation Journal on Education, Engineering, and Agriculture Journal · doi:10.62960/irijeea.v1i2.23 · Cited by 6
“Of the 76 respondents, 38 were subjected to experimental set-up and the remaining half as the control group were taught exploiting lecture method of teaching adopting principles suggested by Gopalan et al., (2020), and Miller et al., (2020) when deciding for experimental inquiry of research.”
Diño, M. J. S., Dion, K. W., Abadir, P. M., Budhathoki, C. B., Huang, C., Padula, W. V., Himmelfarb, C. D., & Davidson, P. M. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1383407 · Cited by 8
Boughdady, A., Elashri, N., & Ibrahim, S. (2024) · Zagazig Nursing Journal · doi:10.21608/znj.2024.343343 · Cited by 6
“The advantages of this research design that we can compare scores post intervention to scores on the same measure in the same participants prior to intervention (22) .”
BackgroundMalli, I. A., Mohamud, M. S., & Al-Nasser, S. (2023) · Antibiotics · doi:10.3390/antibiotics12101546 · Cited by 8
“Even though this study was conducted based on non-random selection, it is the only experimental design that can establish cause-and-effect relationships between an independent and dependent variable [36].”
BackgroundWainwright, K., Mayer, I., Gonçalves, A. S. O., Schulz, R. S., Kiel, S., Chenot, J., Flöel, A., von Podewils, F., Angermaier, A., & Kurth, T. (2023) · BMC Health Services Research · doi:10.1186/s12913-023-09724-w · Cited by 8
“For such interventions in primary care, a CRT with settings as the unit of randomization and patients nested within those settings is particularly suited as it reduces the risk of contamination [28].”
BackgroundThe Effect of Wordwall Based Media on Students' Procedure Text Achievement in Ninth Grade Students
Adnyana, I., & Dewi, K. S. (2022) · Jurnal Pendidikan Bahasa Inggris Undiksha · doi:10.23887/jpbi.v10i1.45143 · Cited by 6
“Typically, the researcher used this experimental design to see if there is a relationship between the two variables or if they have an effect on each other (Miller et al., 2020).”
Methodvan Doorne, I., van Schie, V. M. W., Parlevliet, J., Willems, D., van Rijn, M., & Buurman, B. (2022) · Archives of gerontology and geriatrics (Print) · doi:10.1016/j.archger.2022.104782 · Cited by 6
Bruins, F., van Acht, M. V., Bronckers, I. M., Groenewoud, H., Jong, E. D. D., & Seyger, M. (2022) · Acta Dermato-Venereologica · doi:10.2340/actadv.v102.1000 · Cited by 6
“Therefore we were able to compare FA regimens through a quasi-experimental design (18) with a more recent cohort (November 2016 to October 2020) using FA 6 times per week and a “historical” comparison cohort (September 2008 to November 2016) using FA once weekly from the same registry.”
MethodDuffles, L. F., Menino, A. P., Taira, T. M., de Oliveira, S., Salvador, S. L. D. S., Messora, M. R., Vinolo, M. A. R., & Fukada, S. Y. (2021) · Archives of Oral Biology · doi:10.1016/j.archoralbio.2021.105324 · Cited by 6
Dino, M. J. S., Dion, K. W., Abadir, P. M., Budhathoki, C., Huang, C.-M., Ong, I., Vital, J. C., Cotter, V. T., Himmelfarb, C. R. D., & Davidson, P. M. (2024) · Health Informatics Journal · doi:10.1177/14604582241267793 · Cited by 5
“18 Speci fi cally, a pre-post control group design was adopted 19 to compare the measurements from the control and experimental groups.”
An introduction to implementation science in rehabilitation medicine
Roseen, E. J., Welch, S. A., Stout, N. L., Bean, J. F., Friedly, J. L., & Mittman, B. S. (2025) · PM&R · doi:10.1002/pmrj.70040 · Cited by 5
“87,88 Effectiveness-oriented implementation trials are often conducted in health systems interested in achieving near-term improvements in implementation outcomes in addition to supporting the production of scientific knowledge.”
Metadata
- Authors
- Miller, C. J., Smith, S. N., & Pugatch, M.
- Affiliations
- United States Department of Veterans Affairs (US)
- University of Michigan (US)
- Corresponding author
- Miller, Christopher J.
- Venue
- Psychiatry Research
- Publisher
- Elsevier BV
- Published
- 2020-01
- Volume
- 283
- Pages
- 112452
- ISSN
- 0165-1781 (ISSN-L)
- PMC
- PMC6923620
- OpenAlex
- W2951715721
- Cited by
- 373
- References
- 67
- Citations by year
- 2026: 63 · 2025: 89 · 2024: 67 · 2023: 63 · 2022: 56 · 2021: 20 · 2020: 14
- FWCI
- 4.18
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Psychiatry Research (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health
- U.S. Department of Veterans Affairs
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- implementation research, implementation science, experimental design, quasi-experimental design, randomized controlled trials, interrupted time series, stepped wedge design, pre-post design, research design selection, clinical adoption, intervention sustainability
- MeSH terms
- Implementation Science, Humans, Research Design, Randomized Controlled Trials as Topic, Control Groups, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
