Factorial Experiments
Open accessAbstract
Background An understanding of the individual and combined effects of a set of intervention components is important for moving the science of preventive medicine interventions forward. This understanding can often be achieved in an efficient and economical way via a factorial experiment, in which two or more independent variables are manipulated. The factorial experiment is a complement to the RCT; the two designs address different research questions.Purpose To offer an introduction to factorial experiments aimed at investigators trained primarily in the RCT.Methods The factorial experiment is compared and contrasted with other experimental designs used commonly in intervention science to highlight where each is most efficient and appropriate.Results Several points are made: factorial experiments make very efficient use of experimental subjects when the data are properly analyzed; a factorial experiment can have excellent statistical power even if it has relatively few subjects per experimental condition; and when conducting research to select components for inclusion in a multicomponent intervention, interactions should be studied rather than avoided.Conclusions Investigators in preventive medicine and related areas should begin considering factorial experiments alongside other approaches. Experimental designs should be chosen from a resource management perspective, which states that the best experimental design is the one that provides the greatest scientific benefit without exceeding available resources.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Randomised controlled trials were originally developed to test single-component interventions, but psychological interventions usually consist of multiple treatment components.
“Indeed, RCTs were originally developed to test single-component interventions (eg, drugs), but psychological interventions usually consist of multiple treatment components (eg, mindfulness, psychoeducation and action plans).59”
From Optimising psychological treatment for pain after breast cancer: a factorial design study protocol in Denmark (2023) · cites this paper as [59]
The Abstract (Background and Conclusions; page unavailable) discusses multicomponent interventions and describes treatment-package RCTs as comparing a package of components with a control. The paper does not say that RCTs were originally developed to test single-component interventions, nor does it establish that psychological interventions usually have multiple components or give the cited examples. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe Multiphase Optimization Strategy is a methodological framework designed specifically for optimisation of multicomponent interventions.
“The Multiphase Optimization Strategy (MOST) is a methodological framework, conceptually rooted in engineering, designed specifically for optimisation of multicomponent interventions.”
From Optimising psychological treatment for pain after breast cancer: a factorial design study protocol in Denmark (2023) · cites this paper as [59]
The References section identifies MOST as the “multiphase optimization strategy,” and the Advantages of factorial experiments section discusses choosing components for an optimal multicomponent intervention. The paper does not itself describe MOST as a methodological framework designed specifically for that purpose; its substantive discussion is about factorial experiments. (LLM, conf=0.72)
Experimental — accuracy not yet measuredThe Multiphase Optimization Strategy enables the systematic identification of active treatment components in multicomponent interventions, including how the presence or absence of one or more components affects the performance of other components.
“Specifically, MOST enables the systematic identification of active treatment components in multicomponent interventions, including how the presence or absence of one or more components affects the performance of other components.59–61”
From Optimising psychological treatment for pain after breast cancer: a factorial design study protocol in Denmark (2023) · cites this paper as [59]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredSystem identification approaches obtain dynamic systems models that are the basis for designing control systems achieving optimisation.
“System identification approaches are used in engineering to obtain dynamic systems models; these in turn are the basis for the design of control systems which achieve optimization. 45”
From Evaluating Digital Health Interventions (2016) · cites this paper as [45]
MOST's preparatory and optimization phases are more efficient for intervention development than a traditional randomized controlled trial (RCT) because a traditional RCT would take substantially greater resources (time and sample size per RCT) to meet this goal.
“MOST’s preparatory and optimization phases are more efficient for intervention development than a traditional randomized controlled trial (RCT) since it would take substantially greater resources (time and sample size per RCT) to meet this goal [37].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical Methods for Research Workers
Fisher, R. A. (1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,883 · see citing works
Response Surface Methodology: Process and Product Optimization Using Designed Experiments
Gunst, R. F., Myers, R. H., & Montgomery, D. C. (1996) · Technometrics · doi:10.2307/1270613 · Cited by 9,576 · see citing works
Experimental Design: Procedures for the Behavioral Sciences
Kirk, R. (2013) · doi:10.4135/9781483384733 · Cited by 8,539 · see citing works
Response Surface Methodology: Process and Product Optimization Using Designed Experiments
Gunst, R. F. (1996) · Technometrics · doi:10.1080/00401706.1996.10484509 · Cited by 3,302 · see citing works
Statistical Methods for Research Workers.
I., L., & Fisher, R. A. (1926) · Journal Of The Royal Statistical Society · doi:10.2307/2341488 · Cited by 2,171 · see citing works
A Factorial Trial of Six Interventions for the Prevention of Postoperative Nausea and Vomiting
Apfel, C. C., Korttila, K. T., Abdalla, M., Kerger, H., Turan, A., Vedder, I., Zernak, C., Danner, K., Jokela, R. M., Pocock, S., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejmoa032196 · Cited by 1,550 · see citing works
Design and Analysis of Experiments
Ryan, T. P. (2007) · doi:10.1002/9780470187548.ch12 · Cited by 1,436 · see citing works
Experiments - Planning, Analysis and Parameter Design Optimization
(2001) · Journal of Tropical Pediatrics · doi:10.1093/tropej/47.2.126 · Cited by 1,132 · see citing works
Collins, L. M., Murphy, S. A., & Strecher, V. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.01.022 · Cited by 1,082 · see citing works
Self-citationDesign of Experiments: Statistical Principles of Research Design and Analysis
Kuehl, R. O. (1999) · Cited by 908 · see citing works
Design of Experiments: Statistical Principles of Research Design and Analysis
Anderson, M. J., & Whitcomb, P. J. (2001) · Technometrics · doi:10.1198/tech.2001.s589 · Cited by 657 · see citing works
A strategy for optimizing and evaluating behavioral interventions
Collins, L. M., Murphy, S. A., Nair, V. N., & Strecher, V. J. (2005) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm3001_8 · Cited by 492 · see citing works
Self-citationCollins, L. M., Dziak, J. J., & Li, R. (2009) · Psychological Methods · doi:10.1037/a0015826 · Cited by 431 · see citing works
Self-citationLeBlanc, E. S., O’Connor, E., Whitlock, E. P., Patnode, C. D., & Kapka, T. J. (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-7-201110040-00006 · Cited by 401 · see citing works
The Multiphase Optimization Strategy for Engineering Effective Tobacco Use Interventions
Collins, L. M., Baker, T. B., Mermelstein, R. J., Piper, M. E., Jorenby, D. E., Smith, S. S., Christiansen, B. A., Schlam, T. R., Cook, J. W., & Fiore, M. C. (2011) · Annals of Behavioral Medicine · doi:10.1007/s12160-010-9253-x · Cited by 300 · see citing works
Self-citationWeb-Based Smoking-Cessation Programs
Strecher, V. J., McClure, J. B., Alexander, G. L., Chakraborty, B., Nair, V. N., Konkel, J. M., Greene, S. M., Collins, L. M., Carlier, C. C., Wiese, C. J., et al. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.12.024 · Cited by 266 · see citing works
Self-citationDziak, J. J., Nahum-Shani, I., & Collins, L. M. (2012) · Psychological Methods · doi:10.1037/a0026972 · Cited by 142 · see citing works
Self-citationDeveloping multicomponent interventions using fractional factorial designs
Chakraborty, B., Collins, L. M., Strecher, V. J., & Murphy, S. A. (2009) · Statistics in Medicine · doi:10.1002/sim.3643 · Cited by 107 · see citing works
Self-citationExperiments: Planning, Analysis, and Parameter Design Optimization
Lenth, R. V. (2002) · Journal of the American Statistical Association · doi:10.1198/jasa.2002.s473 · Cited by 82 · see citing works
Overweight and Obesity Prevention for Adolescents
Bonsergent, E., Agrinier, N., Thilly, N., Tessier, S., Legrand, K., Lecomte, E., Aptel, E., Herçberg, S., Collin, J.-F., & Briançon, S. (2012) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2012.09.055 · Cited by 77 · see citing works
Translational research in South Africa: evaluating implementation quality using a factorial design
Caldwell, L. L., Smith, E. A., Collins, L. M., Graham, J. W., Lai, M., Wegner, L., Vergnani, T., Matthews, C., & Jacobs, J. (2012) · Child & Youth Care Forum · doi:10.1007/s10566-011-9164-4 · Cited by 50 · see citing works
Self-citationGray, L. J., Cooper, N. J., Dunkley, A. J., Warren, F. C., Ara, R., Abrams, K. R., Davies, M. J., Khunti, K. K., & Sutton, A. J. (2012) · Obesity Reviews · doi:10.1111/j.1467-789x.2011.00981.x · Cited by 47 · see citing works
Effect coding versus dummy coding in analysis of data from factorial experiments
Kugler, K. C., Trail, J. B., Dziak, J. J., & Collins, L. M. (2012) · Cited by 36 · see citing works
Self-citationMcClure, J. B., Derry, H., Riggs, K. R., Westbrook, E. W., St. John, J., Shortreed, S. M., Bogart, A., & An, L. C. (2012) · Contemporary Clinical Trials · doi:10.1016/j.cct.2012.06.009 · Cited by 33 · see citing works
Wolbers, M., Heemskerk, D., Chau, T. T. H., Yen, N. T. B., Caws, M., Farrar, J., & Day, J. (2011) · Trials · doi:10.1186/1745-6215-12-26 · Cited by 25 · see citing works
Using a Fractional Factorial Design to Increase Direct Mail Response atMother JonesMagazine
Ledolter, J., & Swersey, A. J. (2006) · Quality Engineering · doi:10.1080/08982110600846622 · Cited by 16 · see citing works
Selection in factorial experiments with interaction, especially the 2×2 case
Taneja, B. K., & Dudewicz, E. J. (1987) · Acta Mathematica Sinica English Series · doi:10.1007/bf02560033 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Michie, S., Yardley, L., West, R., Patrick, K., & Greaves, F. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7126 · Cited by 865 · see citing works
Evaluating Digital Health Interventions
Murray, E., Hekler, E. B., Andersson, G., Collins, L. M., Doherty, A., Hollis, C., Rivera, D. E., West, R., & Wyatt, J. C. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.008 · Cited by 847 · see citing works
Self-citationThe Lancet Commission on the future of care and clinical research in autism
Lord, C., Charman, T., Havdahl, A., Carbone, P., Anagnostou, E., Boyd, B., Carr, T., de Vries, P. J., Dissanayake, C., Divan, G., et al. (2022) · The Lancet · doi:10.1016/s0140-6736(21)01541-5 · Cited by 924 · see citing works
Internet-delivered psychological treatments: from innovation to implementation
Andersson, G., Titov, N., Dear, B. F., Rozental, A., & Carlbring, P. (2019) · World Psychiatry · doi:10.1002/wps.20610 · Cited by 747 · see citing works
The impact of digital technology on psychological treatments and their dissemination
Fairburn, C. G., & Patel, V. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.08.012 · Cited by 448 · see citing works
Experimental and quasi-experimental designs in implementation research
Miller, C. J., Smith, S. N., & Pugatch, M. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2019.06.027 · Cited by 373 · see citing works
Inclusion of females does not increase variability in rodent research studies
Beery, A. K. (2018) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2018.06.016 · Cited by 361 · see citing works
Klasnja, P., Smith, S., Seewald, N. J., Lee, A., Hall, K., Luers, B., Hekler, E. B., & Murphy, S. A. (2019) · Annals of Behavioral Medicine · doi:10.1093/abm/kay067 · Cited by 241 · see citing works
Lipschitz, J. M., Pike, C. K., Hogan, T. P., Murphy, S. A., & Burdick, K. E. (2023) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-023-00297-3 · Cited by 230 · see citing works
Collins, L. M., Kugler, K. C., & Gwadz, M. V. (2015) · AIDS and Behavior · doi:10.1007/s10461-015-1145-4 · Cited by 162 · see citing works
Self-citationMoller, A. C., Merchant, G., Conroy, D. E., West, R., Hekler, E., Kugler, K. C., & Michie, S. (2017) · Journal of Behavioral Medicine · doi:10.1007/s10865-016-9818-7 · Cited by 151 · see citing works
Self-citationPrecision Health: The Role of the Social and Behavioral Sciences in Advancing the Vision
Hekler, E. B., Tiro, J. A., Hunter, C. M., & Nebeker, C. (2020) · Annals of Behavioral Medicine · doi:10.1093/abm/kaaa018 · Cited by 147 · see citing works
Schroé, H., Dyck, D. V., Paepe, A. L. D., Poppe, L., Loh, W. W., Verloigne, M., Loeys, T., Bourdeaudhuij, I. D., & Crombez, G. (2020) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-020-01001-x · Cited by 140 · see citing works
Tutorial for Using Control Systems Engineering to Optimize Adaptive Mobile Health Interventions
Hekler, E. B., Rivera, D. E., Martín, C. A., Phatak, S. S., Freigoun, M. T., Korinek, E. V., Klasnja, P., Adams, M. A., & Buman, M. P. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.8622 · Cited by 137 · see citing works
Developing Embodied Conversational Agents for Coaching People in a Healthy Lifestyle: Scoping Review
Kramer, L. L., ter Stal, S., Mulder, B. C., de Vet, E., & van Velsen, L. (2020) · Journal of Medical Internet Research · doi:10.2196/14058 · Cited by 121 · see citing works
“A more fine-grained evaluation can be achieved by means of a factorial design, as this allows researchers to deliver specific intervention components to different groups of users [42].”
BackgroundBot, M., Brouwer, I. A., Roca, M., Kohls, E., Penninx, B. W. J. H., Watkins, E., van Grootheest, G., Cabout, M., Hegerl, U., Gili, M., et al. (2019) · JAMA · doi:10.1001/jama.2019.0556 · Cited by 128 · see citing works
Adams, M. A., Hurley, J. C., Todd, M., Bhuiyan, N., Jarrett, C. L., Tucker, W. J., Hollingshead, K. E., & Angadi, S. S. (2017) · BMC Public Health · doi:10.1186/s12889-017-4197-8 · Cited by 110 · see citing works
Designs and methods for implementation research: Advancing the mission of the CTSA program
Hwang, S., Birken, S. A., Melvin, C. L., Rohweder, C. L., & Smith, J. D. (2020) · Journal of Clinical and Translational Science · doi:10.1017/cts.2020.16 · Cited by 97 · see citing works
Managing temptation in obesity treatment: A neurobehavioral model of intervention strategies
Appelhans, B. M., French, S. A., Pagoto, S., & Sherwood, N. E. (2015) · Appetite · doi:10.1016/j.appet.2015.09.035 · Cited by 99 · see citing works
Piper, M. E., Fiore, M. C., Smith, S. S., Fraser, D. L., Bolt, D. M., Collins, L. M., Mermelstein, R. J., Schlam, T. R., Cook, J. W., Jorenby, D. E., et al. (2015) · Addiction · doi:10.1111/add.13162 · Cited by 89 · see citing works
Self-citationImproving transparency and scientific rigor in academic publishing
Prager, E. M., Chambers, K. E., Plotkin, J. L., McArthur, D. L., Bandrowski, A. E., Bansal, N., Martone, M. E., Bergstrom, H. C., Bespalov, A., & Graf, C. (2019) · Journal of Neuroscience Research · doi:10.1002/jnr.24340 · Cited by 83 · see citing works
Chandler, M. J., Locke, D. E., Crook, J. E., Fields, J. A., Ball, C. T., Phatak, V. S., Dean, P. M., Morris, M., & Smith, G. E. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.3016 · Cited by 75 · see citing works
Gwadz, M. V., Collins, L. M., Cleland, C. M., Leonard, N. R., Wilton, L., Gandhi, M., Braithwaite, R. S., Perlman, D. C., Kutnick, A. H., & Ritchie, A. S. (2017) · BMC Public Health · doi:10.1186/s12889-017-4279-7 · Cited by 72 · see citing works
Self-citationWagner, L. I., Tooze, J. A., Hall, D. L., Levine, B. J., Beaumont, J., Duffecy, J., Victorson, D., Gradishar, W., Leach, J., Saphner, T., et al. (2021) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djab100 · Cited by 67 · see citing works
McAllister, L. S., Pepper, G. V., Virgo, S., & Coall, D. A. (2016) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2015.0151 · Cited by 65 · see citing works
Machine Learning Optimization of Lignin Properties in Green Biorefineries
Löfgren, J., Tarasov, D., Koitto, T., Rinke, P., Balakshin, M. Y., & Todorović, M. (2022) · ACS Sustainable Chemistry & Engineering · doi:10.1021/acssuschemeng.2c01895 · Cited by 65 · see citing works
McMahon, S., Lewis, B. A., Oakes, J. M., Wyman, J. F., Guan, W., & Rothman, A. J. (2017) · Annals of Behavioral Medicine · doi:10.1007/s12160-016-9863-z · Cited by 65 · see citing works
Pervasive Neglect of Sex Differences in Biomedical ResearchPreprint
Zucker, I., Prendergast, B. J., & Beery, A. K. (2021) · Cold Spring Harbor Perspectives in Biology · doi:10.1101/cshperspect.a039156 · Cited by 63 · see citing works
on behalf of the MooDFOOD Prevention Trial Investigators, Roca, M., Kohls, E., Gili, M., Watkins, E. R., Owens, M., Hegerl, U., van Grootheest, G., Bot, M., Cabout, M., et al. (2016) · BMC Psychiatry · doi:10.1186/s12888-016-0900-z · Cited by 63 · see citing works
Research strategies to discern active psychological therapy components: A scoping review
Leijten, P., Weisz, J. R., & Gardner, F. (2021) · Clinical Psychological Science · doi:10.1177/2167702620978615 · Cited by 55 · see citing works
Cook, J. W., Collins, L. M., Fiore, M. C., Smith, S. S., Fraser, D. L., Bolt, D. M., Baker, T. B., Piper, M. E., Schlam, T. R., Jorenby, D. E., et al. (2015) · Addiction · doi:10.1111/add.13161 · Cited by 60 · see citing works
Self-citationImproving transparency and scientific rigor in academic publishing
Prager, E. M., Chambers, K. E., Plotkin, J. L., McArthur, D. L., Bandrowski, A. E., Bansal, N., Martone, M. E., Bergstrom, H. C., Bespalov, A., & Graf, C. (2018) · Brain and Behavior · doi:10.1002/brb3.1141 · Cited by 61 · see citing works
Enhancing the effectiveness of smoking treatment research: conceptual bases and progress
Baker, T. B., Collins, L. M., Mermelstein, R. J., Piper, M. E., Schlam, T. R., Cook, J. W., Bolt, D. M., Smith, S. S., Jorenby, D. E., Fraser, D. L., et al. (2015) · Addiction · doi:10.1111/add.13154 · Cited by 54 · see citing works
Self-citationTargeting self-regulation to promote health behaviors in children
Miller, A. L., Gearhardt, A. N., Fredericks, E. M., Katz, B., Shapiro, L. F., Holden, K., Kaciroti, N., Gonzalez, R., Hunter, C. M., & Lumeng, J. C. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2017.09.008 · Cited by 54 · see citing works
Nazief, A. M., Hassaan, P. S., Khalifa, H. M., Sokar, M. S., & El-Kamel, A. H. (2020) · International Journal of Nanomedicine · doi:10.2147/ijn.s235290 · Cited by 53 · see citing works
A smartphone “app”-delivered randomized factorial trial targeting physical activity in adults
Fanning, J. T., Roberts, S. A., Hillman, C. H., Mullen, S. P., Ritterband, L. M., & McAuley, E. (2017) · Journal of Behavioral Medicine · doi:10.1007/s10865-017-9838-y · Cited by 53 · see citing works
Factorial Designs Help to Understand How Psychological Therapy Works
Watkins, E. R., & Newbold, A. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00429 · Cited by 52 · see citing works
Watkins, E. R., Newbold, A., Tester‐Jones, M., Javaid, M., Cadman, J., Collins, L. M., Graham, J., & Mostazir, M. (2016) · BMC Psychiatry · doi:10.1186/s12888-016-1054-8 · Cited by 51 · see citing works
Self-citationGlass, J. E., McKay, J. R., Gustafson, D. H., Kornfield, R., Rathouz, P. J., Mctavish, F. M., Atwood, A. K., Isham, A., Quanbeck, A., & Shah, D. V. (2017) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2017.03.011 · Cited by 46 · see citing works
Toward Usable Evidence: Optimizing Knowledge Accumulation in HCI Research on Health Behavior Change
Klasnja, P., Hekler, E. B., Korinek, E. V., Harlow, J., & Mishra, S. R. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3026013 · Cited by 44 · see citing works
Farrer, L. M., Gulliver, A., Katruss, N., Fassnacht, D. B., Kyrios, M., & Batterham, P. J. (2019) · Internet Interventions · doi:10.1016/j.invent.2019.100276 · Cited by 46 · see citing works
Improved Designs for Cluster Randomized Trials
Crespi, C. M. (2015) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-032315-021702 · Cited by 45 · see citing works
Health Promotion and Healthier Products Increase Vending Purchases: A Randomized Factorial Trial
Hua, S. V., Kimmel, L., Emmenes, M. V., Taherian, R., Remer, G., Millman, A., & Ickovics, J. R. (2017) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2016.12.006 · Cited by 46 · see citing works
Taylor, C. B., Ruzek, J. I., Fitzsimmons‐Craft, E. E., Sadeh‐Sharvit, S., Topooco, N., Weissman, R. S., Eisenberg, D., Mohr, D. C., Graham, A. K., Jacobi, C., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/17493 · Cited by 46 · see citing works
A Randomized Controlled Trial of an Optimized Smoking Treatment Delivered in Primary Care
Piper, M. E., Cook, J. W., Schlam, T. R., Jorenby, D. E., Smith, S. S., Collins, L. M., Mermelstein, R. J., Fraser, D. L., Fiore, M. C., & Baker, T. B. (2018) · Annals of Behavioral Medicine · doi:10.1093/abm/kax059 · Cited by 46 · see citing works
Self-citationLin, W. Y., Prabhakar, A. K., Mohan, B. C., & Wang, C. (2019) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2019.135289 · Cited by 43 · see citing works
Radomski, A. D., Wozney, L., McGrath, P., Huguet, A., Hartling, L., Dyson, M. P., Bennett, K., & Newton, A. S. (2019) · Journal of Medical Internet Research · doi:10.2196/11128 · Cited by 40 · see citing works
“Modeling, factorial designs or the multiphase optimization strategy [110-112] (see also Baker et al’s study [113] for other suggestions) can be used to evaluate the best set (individual or combination) of program features to use under different conditions (eg, delivery setting and start or end of…”
MethodWatkins, E., Newbold, A., Tester-Jones, M., Collins, L. M., & Mostazir, M. (2023) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2023.1937 · Cited by 40 · see citing works
Self-citationMiller, A. L., Lo, S. L., Bauer, K. W., & Fredericks, E. M. (2020) · Health Psychology Review · doi:10.1080/17437199.2020.1718530 · Cited by 41 · see citing works
Hrynyschyn, R., Prediger, C., Stock, C., & Helmer, S. M. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19095221 · Cited by 43 · see citing works
Sekoai, P. T., Daramola, M. O., Mogwase, B. M. S., Engelbrecht, N., Yoro, K. O., du Preez, S. P., Mhlongo, S. I., Ezeokoli, O. T., Ghimire, A., Ayeni, A. O., et al. (2020) · Biomass and Bioenergy · doi:10.1016/j.biombioe.2020.105673 · Cited by 38 · see citing works
Phillips, S. M., Collins, L. M., Penedo, F. J., Courneya, K. S., Welch, W. A., Cottrell, A., Lloyd, G. R., Gavin, K. L., Cella, D. F., Ackermann, R. T., et al. (2018) · Contemporary Clinical Trials · doi:10.1016/j.cct.2018.01.001 · Cited by 37 · see citing works
Self-citationLi, D. H., Brown, C. H., Gallo, C. G., Morgan, E., Sullivan, P. S., Young, S. D., & Mustanski, B. S. (2019) · Current HIV/AIDS Reports · doi:10.1007/s11904-019-00455-4 · Cited by 36 · see citing works
What is the optimum design for my animal experiment?
Karp, N. A., & Fry, D. (2021) · BMJ Open Science · doi:10.1136/bmjos-2020-100126 · Cited by 39 · see citing works
Public health measures for covid-19
Glasziou, P., Michie, S., & Fretheim, A. (2021) · BMJ · doi:10.1136/bmj.n2729 · Cited by 36 · see citing works
Berg, M., Rozental, A., de Brun Mangs, J., Näsman, M., Strömberg, K., Viberg, L., Wallner, E., Åhman, H., Silfvernagel, K., Zetterqvist, M., et al. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00503 · Cited by 35 · see citing works
Bernstein, S. L., Dziura, J., Weiss, J., Miller, T. R., Vickerman, K. A., Grau, L. E., Pantalon, M. V., Abroms, L. C., Collins, L. M., & Toll, B. A. (2017) · Contemporary Clinical Trials · doi:10.1016/j.cct.2017.12.016 · Cited by 35 · see citing works
Self-citationSlater, J. S., Parks, M. J., Nelson, C. L., & Hughes, K. D. (2018) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-18-0038 · Cited by 34 · see citing works
Effects Of Alternative Food Voucher Delivery Strategies On Nutrition Among Low-Income Adults
Basu, S., Gardner, C. D., White, J. S., Rigdon, J., Carroll, M. M., Akers, M., & Seligman, H. K. (2019) · Health Affairs · doi:10.1377/hlthaff.2018.05405 · Cited by 34 · see citing works
on behalf of the DIAMOND Study Group, Bloomfield, F. H., Harding, J. E., Meyer, M. P., Alsweiler, J. M., Jiang, Y., Wall, C. R., & Alexander, T. (2018) · BMC Pediatrics · doi:10.1186/s12887-018-1195-7 · Cited by 34 · see citing works
Novel adaptive surrogate model based on LRPIM for probabilistic analysis of turbine disc
Mao, J., Hu, D., Li, D., Wang, R., & Song, J. H. (2017) · Aerospace Science and Technology · doi:10.1016/j.ast.2017.07.044 · Cited by 33 · see citing works
Edmunds, S. R., Frost, K. M., Sheldrick, R. C., Bravo, A., Straiton, D., Pickard, K. E., Grim, V., Drahota, A., Kuhn, J., Azad, G. F., et al. (2022) · Autism · doi:10.1177/13623613211064431 · Cited by 33 · see citing works
Developing Optimized Adaptive Interventions in Education
Almirall, D., Kasari, C. L., McCaffrey, D. F., & Nahum-Shani, I. B. (2017) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2017.1407136 · Cited by 32 · see citing works
Multilevel factorial designs with experiment-induced clustering.
Nahum-Shani, I. B., Dziak, J. J., & Collins, L. M. (2017) · Psychological Methods · doi:10.1037/met0000128 · Cited by 30 · see citing works
Self-citationDistinguishing moral hazard from access for high-cost healthcare under insurance
Robertson, C. T., Yuan, A., Zhang, W., & Joiner, K. A. (2020) · PLoS ONE · doi:10.1371/journal.pone.0231768 · Cited by 30 · see citing works
Wells, R. D., Guastaferro, K., Azuero, A., Rini, C., Hendricks, B. A., Dosse, C., Taylor, R., Williams, G. R., Engler, S., Smith, C., et al. (2021) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2020.11.017 · Cited by 28 · see citing works
Klasnja, P., Rosenberg, D. E., Zhou, J., Anau, J., Gupta, A. S., & Arterburn, D. E. (2020) · Translational Behavioral Medicine · doi:10.1093/tbm/ibaa040 · Cited by 28 · see citing works
Behavioral Interventions in Mild Cognitive Impairment (MCI): Lessons from a Multicomponent Program
Levy, S.-A. T., Smith, G. E., Wit, L. D., DeFeis, B. L., Ying, G., Amofa, P., Locke, D. E. C., Shandera‐Ochsner, A. L., McAlister, C. B., Phatak, V. S., et al. (2022) · Neurotherapeutics · doi:10.1007/s13311-022-01225-8 · Cited by 30 · see citing works
A self-control training app to increase self-control and reduce aggression – A full factorial design
Kip, H., Silva, M. C. D., Bouman, Y. H. A., van Gemert-Pijnen, L. J. E. W. C., & Kelders, S. M. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100392 · Cited by 25 · see citing works
Pasha, S., Cahill, P. J., Flynn, J. M. “., Sponseller, P. D., & Newton, P. O. (2018) · Journal of Pediatric Orthopaedics · doi:10.1097/bpo.0000000000001136 · Cited by 26 · see citing works
Wagner, L. I., Duffecy, J., Penedo, F., Mohr, D. C., & Cella, D. (2017) · Cancer · doi:10.1002/cncr.30602 · Cited by 24 · see citing works
Experimental Planning Factorial: A brief Review
Oliveira, M. D., Lima, V. M., Yamashita, S. M. A., Alves, P. S., & Portella, A. C. F. (2018) · International Journal of Advanced Engineering Research and Science · doi:10.22161/ijaers.5.6.28 · Cited by 24 · see citing works
Stanger, C., Kowatsch, T., Xie, H., Nahum-Shani, I. B., Lim-Liberty, F., Anderson, M. A. B., Santhanam, P., Kaden, S. J., & Rosenberg, B. (2021) · JMIR Research Protocols · doi:10.2196/27109 · Cited by 22 · see citing works
Optimizing Implementation in Cancer Control (OPTICC): protocol for an implementation science center
Lewis, C. C., Hannon, P. A., Klasnja, P., Baldwin, L., Hawkes, R. J., Blackmer, J., & Johnson, A. N. (2021) · Implementation Science Communications · doi:10.1186/s43058-021-00117-w · Cited by 23 · see citing works
Zhang, X. K., Zhao, Y., Cheng, C., Li, L., Xiao, M., Zhang, G., & Lu, X. (2020) · Environmental Science and Pollution Research · doi:10.1007/s11356-020-09949-w · Cited by 24 · see citing works
Adams, M. A., Todd, M. W., Angadi, S. S., Hurley, J. C., Stecher, C. D., Berardi, V., Phillips, C. B., McEntee, M. L., Hovell, M. F., & Hooker, S. P. (2021) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2021.09.014 · Cited by 23 · see citing works
Anderson, R. L., DiMeglio, L. A., Mander, A., Dayan, C. M., Linsley, P. S., Herold, K. C., Marinac, M., & Ahmed, S. T. (2022) · Diabetes Care · doi:10.2337/dc22-0308 · Cited by 23 · see citing works
Warren, C. M., Dyer, A. A., Blumenstock, J. A., & Gupta, R. S. (2016) · American Journal of Health Education · doi:10.1080/19325037.2015.1133337 · Cited by 23 · see citing works
Fishbein, D. H., & Dariotis, J. K. (2017) · Prevention Science · doi:10.1007/s11121-017-0851-8 · Cited by 22 · see citing works
Morid, M. A., Sheng, O. R. L., Fiol, G. D., Facelli, J. C., Bray, B. E., & Abdelrahman, S. (2020) · JMIR Medical Informatics · doi:10.2196/14272 · Cited by 22 · see citing works
Comparing the Effects of Single‐ and Multiple‐Component Therapies for Insomnia on Sleep Outcomes
Sidani, S., Epstein, D. R., Fox, M. T., & Collins, L. J. (2019) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.12367 · Cited by 21 · see citing works
Self-citationBest marketing strategy selection using fractional factorial design with analytic hierarchy process
Al-Dawalibi, A., Aldali, I. H., & Alkhayyal, B. (2020) · MethodsX · doi:10.1016/j.mex.2020.100927 · Cited by 21 · see citing works
von Thiele Schwarz, U., Förberg, U., Sundell, K., & Hasson, H. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0496-8 · Cited by 20 · see citing works
Common components analysis: An adapted approach for evaluating programs
Morgan, N. R., Davis, K. D., Richardson, C. B., & Perkins, D. F. (2017) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2017.10.009 · Cited by 20 · see citing works
Glass, J. E., Dorsey, C. N., Beatty, T. C., Bobb, J. F., Wong, E. S., Palazzo, L. G., King, D., Mogk, J. M., Stefanik-Guizlo, K., Idu, A., et al. (2023) · Implementation Science · doi:10.1186/s13012-022-01258-9 · Cited by 20 · see citing works
Shandera‐Ochsner, A. L., Chandler, M. J., Locke, D. E. C., Ball, C. T., Crook, J. E., Phatak, V. S., & Smith, G. E. (2021) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617721000485 · Cited by 19 · see citing works
Schwartz, K. L., Shuldiner, J., Langford, B. J., Brown, K. A., Schultz, S. E., Leung, V., Daneman, N., Tadrous, M., Witteman, H. O., Garber, G. E., et al. (2024) · BMJ · doi:10.1136/bmj-2024-079329 · Cited by 19 · see citing works
Magalhães, B. M. B. D. S., Fernandes, C. S., Santos, C., & Martínez‐Galiano, J. M. (2020) · Cancer Nursing · doi:10.1097/ncc.0000000000000823 · Cited by 21 · see citing works
Wyrick, D. L., Tanner, A. E., Milroy, J. J., Guastaferro, K., Bhandari, S., Kugler, K. C., Thorpe, S. Y., Ware, S., Miller, A. M., & Collins, L. M. (2020) · Journal of American College Health · doi:10.1080/07448481.2020.1790571 · Cited by 20 · see citing works
Self-citationDesigning Interventions to Improve the Health of Women Using the ORBIT Model
Walker, L. O., & Czajkowski, S. M. (2019) · Journal of Obstetric, Gynecologic & Neonatal Nursing · doi:10.1016/j.jogn.2019.02.002 · Cited by 19 · see citing works
Guastaferro, K., & Pfammatter, A. F. (2023) · Journal of Clinical and Translational Science · doi:10.1017/cts.2023.546 · Cited by 19 · see citing works
Allender, S. E., Hayward, J., Gupta, S., Sanigorski, A. J., Rana, S., Seward, H. G., Jacobs, S., & Venkatesh, S. (2020) · npj Digital Medicine · doi:10.1038/s41746-019-0205-y · Cited by 18 · see citing works
Coupling Financial Incentives With Direct Mail in Population-Based Practice
Slater, J. S., Parks, M. J., Malone, M. E., Henly, G. A., & Nelson, C. L. (2016) · Health Education & Behavior · doi:10.1177/1090198116646714 · Cited by 18 · see citing works
Revision of the Wisconsin Smoking Withdrawal Scale: Development of brief and long forms.
Smith, S. S., Piper, M. E., Bolt, D. M., Kaye, J. T., Fiore, M. C., & Baker, T. B. (2021) · Psychological Assessment · doi:10.1037/pas0000978 · Cited by 17 · see citing works
Fitzsimmons‐Craft, E. E., Rackoff, G. N., Shah, J., Strayhorn, J. C., D’Adamo, L., DePietro, B., Howe, C. P., Firebaugh, M., Newman, M. G., Collins, L. M., et al. (2024) · International Journal of Eating Disorders · doi:10.1002/eat.24260 · Cited by 19 · see citing works
Self-citationConnecting low-income smokers to tobacco treatment services
Slater, J. S., Nelson, C. L., Parks, M. J., & Ebbert, J. O. (2015) · Addictive Behaviors · doi:10.1016/j.addbeh.2015.10.013 · Cited by 17 · see citing works
Accounting for Complexity in Home Telemonitoring: A Need for Context-Centred Evidence
Ware, P., Seto, E., & Ross, H. J. (2018) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2018.01.022 · Cited by 17 · see citing works
Pellegrini, C. A., Hoffman, S. A., Collins, L. M., & Spring, B. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.09.001 · Cited by 16 · see citing works
Self-citationWalder, N., Frey, A., Berger, T., & Schmidt, S. J. (2025) · Journal of Medical Internet Research · doi:10.2196/67067 · Cited by 28 · see citing works
Gallis, J. A., Bennett, G. G., Steinberg, D., Askew, S., & Turner, E. L. (2018) · Translational Behavioral Medicine · doi:10.1093/tbm/iby131 · Cited by 16 · see citing works
- Authors
- Collins, L. M., Dziak, J. J., Kugler, K. C., & Trail, J. B.
- Affiliations
- Pennsylvania State University (US)
- Corresponding author
- Collins, Linda M.
- Venue
- American Journal of Preventive Medicine
- Publisher
- Elsevier BV
- Published
- 2014-10
- Volume
- 47
- Issue
- 4
- Pages
- 498-504
- ISSN
- 0749-3797 (ISSN-L)
- PMC
- PMC4171184
- OpenAlex
- W106021680
- Cited by
- 335
- References
- 27
- Citations by year
- 2026: 29 · 2025: 24 · 2024: 33 · 2023: 35 · 2022: 27 · 2021: 36 · 2020: 53 · 2019: 31 · 2018: 26 · 2017: 21 · 2016: 10 · 2015: 10
- FWCI
- 17.17
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- American Journal of Preventive Medicine (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute on Drug Abuse (P50DA010075)
- National Cancer Institute (P50CA143188)
- National Center for Advancing Translational Sciences (KL2TR000126)
- National Institute of Diabetes and Digestive and Kidney Diseases (R01DK097364)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- factorial design, multicomponent intervention, intervention components, interaction effects, statistical power, resource management, experimental design, randomized controlled trials
- MeSH terms
- Humans, Preventive Health Services, Preventive Medicine, Research Design, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review