Single-case experimental designs: a systematic review of published research and current standards

“As Gingerich (1984) pointed out the major difference is that the SCEDs mean difference is standardized using within-case variation instead of between-group variation (Smith, 2012).”

“Published N-of-1 studies in psychology have predominately evaluated N-of-1 data using visual analysis only (McDonald, Quinn, et al., 2017; J. D. Smith, 2012), which has been shown to be unreliable and may not identify small but important effects (Kazdin, 2011; Ninci, Vannest, Willson, & Zhang,…”

“Previous studies suggest that symptom improvement, as reported by clients, should follow the onset of C/TA (e.g., Aschieri & Smith, 2012; J. D. Smith & George, 2012) and that the psychotherapist–client dyad will report an improvement in therapeutic process variables, such as the working alliance…”

“Other study designs could be considered, for instance, singlecase experimental design (SCED) studies that are adapted to rare diseases with a variety of specific and clinical situations (Ganz & Ayres, 2018; KrasnyPacini & Evans, 2018; Smith, 2012; Tate et al., 2008).”

“…referred to the lack of consensus regarding the statistical analysis and meta ‐ analysis of SCEs (see e.g., Cohen, Feinstein, Masuda, & Vowles, 2014; Smith, 2012), which may even reinforce some researchers in just performing a visual analysis and calculating a simple Percentage of Nonoverlapping…”

“To draw conclusions in studies that use SCDs, researchers have traditionally relied upon visual inspection of graphed data (Horner & Swoboda, 2014; Smith, 2012).”

“These idiographic study designs are being explored more frequently within behavioral research (Deshpande et al., 2011; Molenaar & Campbell, 2009; Rivera et al., 2007; Smith, 2012; Timms et al., 2012; Velicer, 2010).”

“, t- and F-test) are often not appropriate to analyze SCEs because assumptions of normality are frequently violated for SCE data, SCE data are often autocorrelated, and these tests are insensitive to trends that occur within a phase (Houle, 2009; Smith, 2012).”

“most single-case experiments, and the last two were similarly identified in a subsequent review by Smith (2012). The second and third authors identified articles that contained at least one single-case graph meeting the following criteria: The initial phase of the graph had to be a baseline condition and include at le…”

“57 Single-case experimental designs also have played roles in medicine, especially in chronic respiratory, dermatologic, digestive, or orthopedic pain conditions for medication initiation and discontinuation studies.”

“In contrast to an experimental group design, in which one group is compared with another, participants in a single-subject experiment research provide their own control data for the purpose of comparison in a within-subject, rather than a between-subject, design (65).”

“Methods of assessment in n-of-1 trials have been the subject of ongoing debates in the past years.(48) However, although debates regarding the appropriateness and accuracy of different methods persist, recent developments in this field of research have suggested that the use of statistical analyses, when combined with…”

“Given that this was a proof-of-concept study, the number of participants was based on earlier studies using similar SCED methodologies (Ebbinghaus, 2005; Smith, 2012; Smith and Little, 2018; Bossenbroek et al., 2020).”

“However, a recently completed systematic review of 409 recently published single-case experiments by Smith (2011) suggests that baseline periods have increased since the Huitema review of 1985: The Smith review found a mean number of 10.22 observations in the baseline phase.”

“With respect to assessment of single-case studies methodological quality, this review used the SCED scale, which was considered to be Bthe only psychometrically validated tool for assessing the rigor of SCED methodology^ (Smith 2012, p. 512).”

“In fact, psychologists and behavioral scientists have developed a variety of ideographic analytical approaches for single-case experimental designs [14,15,16].”

“Research shows that the difference of means is one of the best analysis methods for self-experiments due to its simplicity [12, 47].”

“Among these designs, researchers have found that multiple baseline designs were the most frequently used (Coon & Rapp, 2018; Shadish & Sullivan, 2011; Smith, 2012).”

“Usually, the comparison takes place between different time periods, in which a collected baseline phase is compared with subsequent phases (Smith, 2012).”

“Visual analysis remains the standard by which most single case experimental designs are analysed (Smith, 2012).”

“…growing focus on evidence-based practice along with the rising status of SCD research (Plavnick and Ferreri 2013; Shadish 2014) has resulted in numerous attempts to develop reliable, objective, and defensible standards that can be used to evaluate SCD evidence (see Maggin et al. 2014; Smith 2012).”

“Data for both groups were primarily evaluated using visual analysis during the conduct of the study; however, a supplemental statistical analysis was also conducted using a paired-sample t test following completion of the study (Kazdin, 2011; Smith, 2012).”

“Missing data are common in SCEs, often due to human factors such as forgetfulness or systematic noncompliance to the protocols of data collection in an experiment (Smith, 2012).”

“The application of randomization is a critical component of SCEDs where threats to internal validity can be experimentally controlled and opportunities for statistical testing can be facilitated.(40) The use of randomization further enables the researcher to determine whether there is a causal or functional relationsh…”

“Recently, studies employing single-case research methods appear to be even more prevalent (Smith, 2012).”

“For single case designs, referring to guidance such as Tate et al. (2008) and Smith (2012) will add to the quality and strength of research evidence.”

“timethod and multisource evaluation of outcome (Smith, 2012), such as HSCED.”

“SCEDs, also referred to as within-subject designs (Ma, 2011), were primarily used to examine the effectiveness of interventions (Krasny-Pacini & Evans, 2018; Smith, 2012).”

“Experimental single-case designs with repeated measurement (e.g., a time-series) are a valuable tool to investigate the effects of therapeutic interventions (e.g., Borckardt et al., 2008) and have been used extensively over the past decade (Smith, 2012).”

“More relevant to special education, scholars have used research synthesis methods to evaluate the methodological characteristics of published SCDs against existing standards and bestpractice recommendations (Shadish & Sullivan, 2011; Smith, 2012) and to examine the extent to which data from SCDs meet the assumptions r…”

“Research strategies are evolving for such single-case experimental designs (e.g., Barlow & Hayes, 1979; Barlow, Nock, & Hersen, 2008; Smith, 2012), but, with interesting exceptions (e.g., Tryon, 1982; Unicomb, Colyvas, Harrison, & Hewat, 2015) statistical analysis of them lags (Shadish, Rindskopf,…”

“What is typically missing from this research is a careful consideration of specific factors unique to a particular patient, clinician, or dyad that may explain differential responses to treatments (Os, Delespaul, Wigman, Myin-Germeys, & Wichers, 2013; Smith, 2012).”

“The evaluation of single-case graphs using visual analysis is widely used and accepted (Kennedy, 2005; Ledford & Gast, 2018; Smith, 2012; WWC, 2017).”

“…of the previous studies used time series statistical analyses to examine intervention effects over time, which has the advantage of evaluating effectiveness across phases of treatment and enhances interpretation of findings compared to visual inspection alone (Borckardt et al., 2008; Smith, 2012).”

“Systematic collection of individualized data can reveal the course (trajectory) and level of change over time (Borckardt et al., 2008; Smith, 2012).”

“SCED can also be used to evaluate rehabilitation efficacy [53].”

“…during a short, but very intensive, measurement period, for example, one or 2 weeks in which a participant is prompted 10 or more times per day to provide self-reports of activities, emotional, psychological or physiological states (Trull and Ebner-Priemer, 2009; Smith, 2012; Liao et al., 2016).”

“Effect size SCED studies play an important role in establishing and confirming evidence-based interventions (Horner et al., 2005; Shadish & Sullivan, 2011; Smith, 2012).”

“Several previous studies have evaluated the extent to which different designs are used in behavioral and special education journals (Hammond & Gast, 2010) and in education and psychology research (Shadish & Sullivan, 2011; Smith, 2012).”

“SCEDs are used across various research fields, including occupational therapy, special education, and rehabilitation (Lane et al., 2017; Ritter et al., 2018; Smith, 2012).”

“Among the many possibilities, one interesting experimental design to explore is the single-case experimental designs (e.g., Smith, 2012).”

“to the literature-base related to reporting characteristics in systematic reviews and meta-analyses involving SCED studies (Beretvas & Chung, 2008; Jamshidi et al., 2017; Schlosser et al., 2008; Smith, 2012).”

“However, as we did not use an xperimental design we could not exclude further threats to the validity of our findings (e.g. regression to mean, placebo etc.; Smith, 2012).”

“…(i.e., multiple-baseline, withdrawal/reversal, and changing criterion designs), the first of which are the most commonly used SCEDs, according to several reviews of published literature (Pustejovsky, Swan, & English, 2023a, 2023b; Shadish & Sullivan, 2011; Smith, 2012; Tanious & Onghena, 2021).”

“The What Works Clearinghouse guidelines for SCEDs (Kratochwill et al., 2010) recommend inspecting six features of the data when performing visual analysis: level, trend, variability, immediacy of the effect, overlap, and CONsistency of DAta Patterns (CONDAP) across similar phases.”

“Based on several meta-analyses, approximately 10% of overall single subject research used alternating treatments design (Hammond & Gast, 2010; Shadish & Sullivan, 2011; Smith, 2012).”

“…interventions, as they are delivered in clinical practice (Borckardt et al., 2008), case by case individual differences in treatment responsiveness (Howard, Moras, Brill, Martinovich, & Lutz, 1996), and the efficacy of single case experimental designs on treatment adherence (Smith, 2012).”

“The second set of standards is concerned with measurement and assessment of the outcome behavior (Kratochwill & Levin, 2014; Smith, 2012).”

“SCED is a rigorous method for developing evidence of the efficacy of an intervention by using participants as their own controls via repeated measures (Smith, 2013).”

“Single-case experimental designs (SCEDs) are strategies capable of meeting criteria for experimental quality (Smith, 2012) and useful for identifying evidence-based practices (Schlosser, 2009).”

“Though single case experimental research designs allow for flexibility of implementing and analyzing intervention research (Smith 2012), a larger sample size would be needed to extend the findings to a larger population of students.”

“Meditation and yoga (de fi ned as a combination of meditation and its context) [1], have gained popularity in contemporary sci-enti fi c research and have been used for several mental health (stress, anxiety, depressive disorders, etc.) [2,3] and physical conditions (pain, etc.) [4,5].”

“We included single-subject studies if the authors reported at least three replications of treatment effects (e.g., ABAB, or multiple baseline designs, see Smith, 2012 for descriptions).”

“Because the importance of SCED research in establishing and confirming evidence-based practices has been increasingly affirmed (Horner et al., 2005; Shadish & Sullivan, 2011; Smith, 2012), it is imperative that SCED research be conducted at the highest level of rigor to yield credible and generalizable results.”

“Sci. 2021, 11, 76. https://doi.org/10.3390/educsci11020076 https://www.mdpi.com/journal/education In a review of SCD standards and 409 empirical studies published in refereed journals between 2000 and 2010, Smith [11] lamented, “SCDs undeniably present researchers with a complex array of methodological and research de…”

“…has focused on the effectiveness of TA with different types of problems, i.e., self-harm (Ougrin et al. 2008); internalizing symptoms, (Aschieri and Smith 2012); externalizing symptoms (Smith et al. 2010); disorganized attachment (Smith and George 2012); and developmental trauma (Tarocchi et al.…”

“More than 400 studies have been conducted using SSRDs [33].”

“…longitudinal inferences on the level of an individual participant, methodological challenges such as low statistical power for inferential statistics, and dif fi culties with interpreting single-subject effect sizes [21], often mean that generalisation to a broader population needs to be considered.”

“…that Strengthen Internal and/or External Validity Intervention researchers often rely on MBD and ATD to make claims about functional relationships (Ledford et al., 2019; Pustejovsky et al., 2019; Shadish & Sullivan, 2011; Smith, 2012; Tanious & Onghena, 2020) even when RAD might be suitable.”

“Unlike traditional single-case experimental designs (Smith, 2012), ABA team members intervened on dormitory staff members as a group. In particular, each staff member’s praise statements, reprimands, and instructions contributed to the total occurrence of behaviors within an observation. In particular, we recorded pra…”

“Multiple researchers have recently compared these standards (e.g., Maggin et al., 2014; Smith, 2012; Wendt & Miller, 2012).”

“Perhaps because of these findings, in single-subject designs, the focus has been shifting toward more formal statistical analyses (Fisch, 2001; Manolov, Gast, Perdices, & Evans, 2014; Smith, 2012), or at the very least toward formal quantifications of visual depictions (Lane & Gast, 2014).”

“, replication of effects across a minimum of three conditions), which was demonstrated in the current study (Smith, 2012).”

“The data collection and analysis were conducted using a single-case ABA design ( Kazdin, 1982 ), chosen for its fl exibility and applicability, and have been used extensively over the past decade ( Smith, 2012 ).”

“As described in the methodical review of Smith (2013) and applied in the study of Kwak et al. (2018), both groups received the intervention and the treatment-as-usual control condition.”

“ically using an interrupted time-series-type analysis to determine the impact [91].”

“Despite the existence of visual aids and randomization procedures and tests (Kratochwill & Levin, 2010), we consider that, in case either of the procedures proves to perform well, such a procedure would be potentially useful given that (a) they are not only visual aids but also offer quantifications that can later be…”