Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy
Open accessAbstract
There is growing interest in how best to adapt and readapt treatments to individuals to maximize clinical benefit. In response, adaptive treatment strategies (ATS), which operationalize adaptive, sequential clinical decision making, have been developed. From a patient's perspective an ATS is a sequence of treatments, each individualized to the patient's evolving health status. From a clinician's perspective, an ATS is a sequence of decision rules that input the patient's current health status and output the next recommended treatment. Sequential multiple assignment randomized trials (SMART) have been developed to address the sequencing questions that arise in the development of ATSs, but SMARTs are relatively new in clinical research. This article provides an introduction to ATSs and SMART designs. This article also discusses the design of SMART pilot studies to address feasibility concerns, and to prepare investigators for a full‐scale SMART. We consider an example SMART for the development of an ATS in the treatment of pediatric generalized anxiety disorders. Using the example SMART, we identify and discuss design issues unique to SMARTs that are best addressed in an external pilot study prior to the full‐scale SMART. We also address the question of how many participants are needed in a SMART pilot study. A properly executed pilot study can be used to effectively address concerns about acceptability and feasibility in preparation for (that is, prior to) executing a full‐scale SMART. Copyright © 2012 John Wiley & Sons, Ltd.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The key to avoiding information bias in a SMART is to clearly separate measures used for treatment, such as tailoring variables, from measures used for research outcomes.
“As with any randomized trial, the key to avoiding this type of bias in a SMART is to make a clear distinction between the measures used for treatment (e.g., tailoring variables) versus the outcomes used for research [80].”
From Introduction to SMART designs for the development of adaptive interventions: with application to weight loss research (2014) · cites this paper as [80]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(1976) · Medical Entomology and Zoology · Cited by 8,899 · see citing works
A tutorial on pilot studies: the what, why and how
Thabane, L., Ma, J., Chu, R., Cheng, J., Ismaila, A., Rios, L. P., Robson, R., Thabane, M., Giangregorio, L., & Goldsmith, C. H. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-1 · Cited by 3,384 · see citing works
Design and analysis of pilot studies: recommendations for good practice
Lancaster, G. A., Dodd, S., & Williamson, P. R. (2004) · Journal of Evaluation in Clinical Practice · doi:10.1111/j..2002.384.doc.x · Cited by 2,572 · see citing works
The role and interpretation of pilot studies in clinical research
Leon, A. C., Davis, L. L., & Kraemer, H. C. (2011) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2010.10.008 · Cited by 1,917 · cited 2× · see citing works
“Scientists preparing to execute a randomized trial should also refer to the literature on executing external pilot studies for pointers on more general uses of pilot studies [34–36].”
Kraemer, H. C., Stice, E., Kazdin, A., Offord, D., & Kupfer, D. (2001) · American Journal of Psychiatry · doi:10.1176/appi.ajp.158.6.848 · Cited by 1,712 · see citing works
Birmaher, B., & Brent, D. (2007) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/chi.0b013e318145ae1c · Cited by 1,056 · see citing works
Optimal Dynamic Treatment Regimes
Murphy, S. A. (2003) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/1467-9868.00389 · Cited by 988 · see citing works
Self-citationDictionary of Statistics and Methodology: A Nontechnical Guide for the Social Sciences
Vaidyanathan, R., & Vogt, W. P. (1994) · Journal of Marketing Research · doi:10.2307/3152235 · Cited by 880 · see citing works
. Caution regarding the use of pilot studies to guide power calculations for study proposals.
Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., & Yesavage, J. A. (2006) · Archives of General Psychiatry · doi:10.1001/archpsyc.63.5.484 · Cited by 702 · see citing works
An experimental design for the development of adaptive treatment strategies
Murphy, S. A. (2005) · Statistics in Medicine · doi:10.1002/sim.2022 · Cited by 595 · see citing works
Self-citationConnolly, S. D., & Bernstein, G. A. (2007) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/01.chi.0000246070.23695.06 · Cited by 548 · see citing works
Optimal Structural Nested Models for Optimal Sequential Decisions
Robins, J. M. (2004) · Lecture notes in statistics · doi:10.1007/978-1-4419-9076-1_11 · Cited by 547 · see citing works
Bayesian Adaptive Methods for Clinical Trials
Berry, S. M., Carlin, B. P., Lee, J. J., & Muller, P. (2010) · CRC Press eBooks · doi:10.1201/ebk1439825488 · Cited by 520 · see citing works
A strategy for optimizing and evaluating behavioral interventions
Collins, L. M., Murphy, S. A., Nair, V. N., & Strecher, V. J. (2005) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm3001_8 · Cited by 492 · see citing works
Self-citationInsel, T. R. (2009) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2008.540 · Cited by 459 · see citing works
Stone, R. M., Berg, D. T., George, S. L., Dodge, R. K., Paciucci, P. A., Schulman, P., Lee, E. J., Moore, J. O., Powell, B. L., & Schiffer, C. A. (1995) · New England Journal of Medicine · doi:10.1056/nejm199506223322503 · Cited by 447 · see citing works
The role of internal pilot studies in increasing the efficiency of clinical trials
Wittes, J. T., & Brittain, E. (1990) · Statistics in Medicine · doi:10.1002/sim.4780090113 · Cited by 440 · see citing works
Marginal Mean Models for Dynamic Regimes
Murphy, S. A., van der Laan, M. J., Robins, J. M., & Conduct Problems Prevention Research Group (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382327 · Cited by 393 · see citing works
Self-citationComparison of Dynamic Treatment Regimes via Inverse Probability Weighting
Hernán, M. Á., Lanoy, É., Costagliola, D. G., & Robins, J. M. (2006) · Basic & Clinical Pharmacology & Toxicology · doi:10.1111/j.1742-7843.2006.pto_329.x · Cited by 281 · see citing works
Cain, L. E., Robins, J. M., Lanoy, É., Logan, R. W., Costagliola, D. G., & Hernán, M. Á. (2010) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1212 · Cited by 233 · see citing works
Dynamic treatment regimes: practical design considerations
Lavori, P. W., & Dawson, R. (2004) · Clinical Trials · doi:10.1191/1740774s04cn002oa · Cited by 229 · see citing works
Demystifying Optimal Dynamic Treatment Regimes
Moodie, E. E. M., Richardson, T. S., & Stephens, D. A. (2007) · Biometrics · doi:10.1111/j.1541-0420.2006.00686.x · Cited by 229 · see citing works
Developing adaptive treatment strategies in substance abuse research
Murphy, S. A., Lynch, K. G., Oslin, D., McKay, J. R., & TenHave, T. (2007) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2006.09.008 · Cited by 211 · see citing works
Self-citationSchneider, L. S. S., Tariot, P. N., Lyketsos, C. G., Dagerman, K., Davis, K. L., Thomas, S., Hsiao, J., Jeste, D. V., Katz, I. R., Olin, J. T., et al. (2001) · American Journal of Geriatric Psychiatry · doi:10.1097/00019442-200111000-00004 · Cited by 211 · see citing works
Orellana, L., Rotnitzky, A., & Robins, J. M. (2010) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1200 · Cited by 202 · see citing works
Bayesian Adaptive Methods for Clinical Trials
Hamilton, C. (2010) · Journal of Statistical Software · doi:10.18637/jss.v037.b01 · Cited by 178 · see citing works
Strengthening clinical effectiveness trials: Equipoise-stratified randomization
Lavori, P. W., Rush, A. J., Wisniewski, S. R., Alpert, J. E., D, M. F. M., Kupfer, D. J., Nierenberg, A. A., Quitkin, F. M., Sackeïm, H. A., Thase, M. E., et al. (2001) · Biological Psychiatry · doi:10.1016/s0006-3223(01)01223-9 · Cited by 153 · see citing works
Q-learning: A data analysis method for constructing adaptive interventions.
Nahum-Shani, I., Qian, M., Almirall, D., Pelham, W. E., Gnagy, B., Fabiano, G. A., Waxmonsky, J. G., Yu, J., & Murphy, S. A. (2012) · Psychological Methods · doi:10.1037/a0029373 · Cited by 141 · see citing works
Self-citationFlexible treatment strategies in chronic disease: clinical and research implications
Lavori, P. W., Dawson, R., & Rush, A. J. (2000) · Biological Psychiatry · doi:10.1016/s0006-3223(00)00946-x · Cited by 138 · see citing works
Adaptive Treatment Strategies in Chronic Disease
Lavori, P. W., & Dawson, R. (2008) · Annual Review of Medicine · doi:10.1146/annurev.med.59.062606.122232 · Cited by 113 · see citing works
Texas Medication Algorithm Project, Phase 3 (TMAP-3)
Rush, A. J., Crismon, M. L., Kashner, T. M., Toprac, M. G., Carmody, T. J., Trivedi, M. H., Suppes, T. M., Miller, A. L., Biggs, M. M., Shores‐Wilson, K., et al. (2003) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v64n0402 · Cited by 108 · see citing works
Wahed, A. S., & Tsiatis, A. A. (2004) · Biometrics · doi:10.1111/j.0006-341x.2004.00160.x · Cited by 103 · see citing works
Strategies for comparing treatments on a binary response with multi-centre data
Agresti, A., & Hartzel, J. S. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000430)19:8 · Cited by 102 · see citing works
Customizing treatment to the patient: Adaptive treatment strategies
Murphy, S. A., Collins, L. M., & Rush, A. J. (2007) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2007.02.001 · Cited by 94 · see citing works
Self-citationEncyclopedia of medical decision making
(2010) · Choice Reviews Online · doi:10.5860/choice.47-2927 · Cited by 93 · see citing works
History-Adjusted Marginal Structural Models and Statically-Optimal Dynamic Treatment Regimens
van der Laan, M. J., Petersen, M. L., & Joffe, M. M. (2005) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1003 · Cited by 73 · see citing works
Orellana, L., Rotnitzky, A., & Robins, J. M. (2010) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1242 · Cited by 53 · see citing works
Screening Experiments for Developing Dynamic Treatment Regimes
Murphy, S. A., & Bingham, D. (2009) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.0119 · Cited by 41 · see citing works
Self-citationDawson, R., & Lavori, P. W. (2004) · Statistics in Medicine · doi:10.1002/sim.1920 · Cited by 41 · see citing works
Bembom, O., & van der Laan, M. J. (2008) · Statistics in Medicine · doi:10.1002/sim.3268 · Cited by 35 · see citing works
Improving the efficiency of estimation in randomized trials of adaptive treatment strategies
Lavori, P. W., & Dawson, R. (2007) · Clinical Trials · doi:10.1177/1740774507081327 · Cited by 20 · see citing works
History-Adjusted Marginal Structural Models and Statically-Optimal Dynamic Treatment RegimesPreprint
van der Laan, M. J., & Petersen, M. L. (2004) · RePEc: Research Papers in Economics · Cited by 3 · see citing works
Buyse, M. (1999) · doi:10.1002/(sici)1097-0258(19991230)18:24
Flagged first, then by citations
Research works printed but not linked: 1
- [39]Zucker DM Wittes JT Schabenberger O Brittain E Internal pilot studies II: comparison of various procedures Statistics in Medicine 1999 18 3493 3509 10611621
Cited by
Sorted by citations
Using Measurement-Based Care to Enhance Any Treatment
Scott, K., & Lewis, C. C. (2015) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2014.01.010 · Cited by 461 · see citing works
Almirall, D., Nahum-Shani, I., Sherwood, N. E., & Murphy, S. A. (2014) · Translational Behavioral Medicine · doi:10.1007/s13142-014-0265-0 · Cited by 453 · see citing works
Self-citationExperimental and quasi-experimental designs in implementation research
Miller, C. J., Smith, S. N., & Pugatch, M. (2020) · Psychiatry Research · doi:10.1016/j.psychres.2019.06.027 · Cited by 373 · see citing works
“Researchers interested in optimizing sequences of implementation strategies that adapt to ongoing needs over time may be interested in a variant of factorial designs known as the sequential, multiple-assignment randomized trial (SMART; Almirall et al., 2012; Collins et al., 2014b; Kilbourne et al., 2014b; Lei et al.,…”
BackgroundBeyond linear mediation: Toward a dynamic network approach to study treatment processes
Hofmann, S. G., Curtiss, J. E., & Hayes, S. C. (2020) · Clinical Psychology Review · doi:10.1016/j.cpr.2020.101824 · Cited by 232 · see citing works
Reinforcement learning for clinical decision support in critical care: comprehensive review
Liu, S., See, K. C., Ngiam, K. Y., Celi, L. A., Sun, X., & Feng, M. (2020) · Journal of Medical Internet Research · doi:10.2196/18477 · Cited by 225 · see citing works
Hermes, E. D., Lyon, A. R., Schueller, S. M., & Glass, J. E. (2019) · Journal of Medical Internet Research · doi:10.2196/11752 · Cited by 149 · see citing works
Kilbourne, A. M., Almirall, D., Eisenberg, D., Waxmonsky, J., Goodrich, D. E., Fortney, J. C., Kirchner, J. E., Solberg, L. I., Main, D., Bauer, M. S., et al. (2014) · Implementation Science · doi:10.1186/s13012-014-0132-x · Cited by 130 · see citing works
Self-citationCzyz, E. K., King, C. A., Prouty, D., Micol, V. J., Walton, M., & Nahum‐Shani, I. (2021) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13383 · Cited by 54 · see citing works
Smith, S. N., Almirall, D., Prenovost, K., Liebrecht, C., Kyle, J., Eisenberg, D., Bauer, M. S., & Kilbourne, A. M. (2019) · Medical Care · doi:10.1097/mlr.0000000000001138 · Cited by 40 · see citing works
Self-citationHaque, S. S., & Freeman, M. C. (2021) · Environmental Health Perspectives · doi:10.1289/ehp7762 · Cited by 48 · see citing works
Wang, C. H., Mazursky-Horowitz, H., & Chronis-Tuscano, A. (2014) · Current Psychiatry Reports · doi:10.1007/s11920-014-0474-8 · Cited by 39 · see citing works
Pistorello, J., Jobes, D. A., Compton, S. N., Locey, N. S., Walloch, J. C., Gallop, R., Au, J. S., Noose, S. K., Young, M., Johnson, J., et al. (2018) · Archives of Suicide Research · doi:10.1080/13811118.2017.1392915 · Cited by 29 · see citing works
Cully, J. A., Hundt, N. E., Fletcher, T., Sansgiry, S., Zeno, D., Kauth, M. R., Kunik, M. E., & Sorocco, K. (2024) · Psychiatric Services · doi:10.1176/appi.ps.20220582 · Cited by 5 · see citing works
- Authors
- Almirall, D., Compton, S. N., Gunlicks‐Stoessel, M., Duan, N., & Murphy, S. A.
- Affiliations
- University of Michigan (US)
- Duke Medical Center (US)
- University of Minnesota (US)
- New York State Psychiatric Institute (US)
- Corresponding author
- Almirall, Daniel
- Venue
- Statistics in Medicine
- Publisher
- Wiley
- Published
- 2012-03-22
- Volume
- 31
- Issue
- 17
- Pages
- 1887-1902
- ISSN
- 0277-6715 (ISSN-L)
- 1097-0258
- PMC
- PMC3399974
- OpenAlex
- W2019384544
- Cited by
- 223
- References
- 43
- Citations by year
- 2026: 17 · 2025: 14 · 2024: 18 · 2023: 17 · 2022: 19 · 2021: 21 · 2020: 15 · 2019: 18 · 2018: 11 · 2017: 15 · 2016: 15 · 2015: 16 · 2014: 17 · 2013: 7 · 2012: 2
- FWCI
- 12.84
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- Statistics in Medicine (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Deep Blue (University of Michigan) (submittedVersion)
- Funders
- Pennsylvania State University
- National Institute of Mental Health (R01MH080015, K23-MH-075843-04, K23MH090216, K23MH075843)
- National Institute on Drug Abuse (P50DA010075)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- adaptive treatment strategies, sequential multiple assignment randomized trial, smart design, pilot study design, feasibility assessment, sample size determination, sequential decision rules, personalized treatment
- MeSH terms
- Anxiety Disorders, Child, Humans, Pilot Projects, Cognitive Behavioral Therapy, Randomized Controlled Trials as Topic, Sertraline
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
