Analyzing data from single-case designs using multilevel models: New applications and some agenda items for future research.
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Several authors have proposed the use of multilevel models to analyze data from single-case designs. This article extends that work in 2 ways. First, examples are given of how to estimate these models when the single-case designs have features that have not been considered by past authors. These include the use of polynomial coefficients to model nonlinear change, the modeling of counts (Poisson distributed) or proportions (binomially distributed) as outcomes, the use of 2 different ways of modeling treatment effects in ABAB designs, and applications of these models to alternating treatment and changing criterion designs. Second, issues that arise when multilevel models are used for the analysis of single-case designs are discussed; such issues can form part of an agenda for future research on this topic. These include statistical power and assumptions, applications to more complex single-case designs, the role of exploratory data analyses, extensions to other kinds of outcome variables and sampling distributions, and other statistical programs that can be used to do such analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Observational study of behavior: sampling methods
Altmann, J. (1974) · Behaviour · doi:10.1163/156853974x00534 · Cited by 13,891 · cited 4×
“Altmann (1974), a primatologist rather than a SCD researcher, provided an informative comparison of the benefits and problems associated with different behavior sampling methods. Mann, Ten Have, Plunkett, and Meisels (1991) discussed different approaches to time sampling to estimate frequency or proportions of time th…”
BackgroundInfluentialApplied Longitudinal Data Analysis: Modeling Change and Event Occurrence
Singer, J. D., & Willett, J. B. (2003) · doi:10.1093/acprof:oso/9780195152968.001.0001 · Cited by 10,647
Bayesian model selection in social research
Raftery, A. E. (1995) · Sociological Methodology · doi:10.2307/271063 · Cited by 6,823
Agresti, A. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/0471249688 · Cited by 6,746
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 2×
“Third, just as is often the case in other literatures (e.g., Francis, 2012; Ioannidis, 2005, 2008; Ioannidis & Lau, 2001; Ioannidis & Panagiotou, 2011; Ioannidis & Trikalinos, 2007; Kyzas, Loizou, & Ioannidis, 2005; Renkewitz, Fuchs, & Fiedler, T hi s do cu m en t is co py ri gh te d by th e A m er…”
Effect Size in Single-Case Research: A Review of Nine Nonoverlap Techniques
Parker, R. I., Vannest, K. J., & Davis, J. L. (2011) · Behavior Modification · doi:10.1177/0145445511399147 · Cited by 841
An exploratory test for an excess of significant findings
Ioannidis, J. P., & Trikalinos, T. A. (2007) · Clinical Trials · doi:10.1177/1740774507079441 · Cited by 751 · cited 1×
“…just as is often the case in other literatures (e.g., Francis, 2012; Ioannidis, 2005, 2008; Ioannidis & Lau, 2001; Ioannidis & Panagiotou, 2011; Ioannidis & Trikalinos, 2007; Kyzas, Loizou, & Ioannidis, 2005; Renkewitz, Fuchs, & Fiedler, T hi s do cu m en t is co py ri gh te d by th e A m er…”
Statistical analysis and optimal design for cluster randomized trials.
Raudenbush, S. W. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.2.173 · Cited by 636
Single-case experimental designs: a systematic review of published research and current standards
Smith, J. D. (2012) · Psychological Methods · doi:10.1037/a0029312 · Cited by 630 · cited 3×
“Both Shadish and Sullivan (2011) and Smith (2012), for example, found that the average SCD had more cases and more time points than the minimum required by the SCD.”
BackgroundMethodStandard Errors and Sample Sizes for Two-Level Research
Snijders, T. A. B., & Bosker, R. J. (1993) · Journal of Educational Statistics · doi:10.3102/10769986018003237 · Cited by 487 · cited 3×
“…no t to be di ss em in at ed br oa dl y. in which persons are nested within some aggregate like classes (Berkhof & Snijders, 2001), often in the context of a grouprandomized trial to test treatment effects (Raudenbush, 1997; Raudenbush & Liu, 2001; Snijders & Bosker, 1993; Zhang & Willson, 2006).”
BackgroundRaudenbush, S. W., & Liu, X.-F. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.4.387 · Cited by 455 · cited 2×
“…no t to be di ss em in at ed br oa dl y. in which persons are nested within some aggregate like classes (Berkhof & Snijders, 2001), often in the context of a grouprandomized trial to test treatment effects (Raudenbush, 1997; Raudenbush & Liu, 2001; Snijders & Bosker, 1993; Zhang & Willson, 2006).”
BackgroundN-of-1 Trials in the Medical Literature: A Systematic Review
Gabler, N. B., Duan, N., Vohra, S., & Kravitz, R. L. (2011) · Medical Care · doi:10.1097/mlr.0b013e318215d90d · Cited by 226
A multilevel meta-analysis of single-subject experimental design studies
Van den Noortgate, W., & Onghena, P. (2008) · Evidence-Based Communication Assessment and Intervention · doi:10.1080/17489530802505362 · Cited by 192
The Aggregation of single-case results using hierarchical linear models
Van den Noortgate, W., & Onghena, P. (2007) · The Behavior Analyst Today · doi:10.1037/h0100613 · Cited by 86 · cited 2×
“Van den Noortgate and Onghena (2007) showed how multilevel models can be applied to three-phase (ABC) designs where a baseline phase is followed by a first treatment and then a second treatment.”
BackgroundMethod
Cited by
Shadish, W. R., Hedges, L. V., & Pustejovsky, J. E. (2014) · Journal of School Psychology · doi:10.1016/j.jsp.2013.11.005 · Cited by 269
Self-citationAnalyzing data from single-case alternating treatments designs.
Manolov, R., & Onghena, P. (2018) · Psychological Methods · doi:10.1037/met0000133 · Cited by 70
A Priori Justification for Effect Measures in Single-Case Experimental Designs
Manolov, R., Moeyaert, M., & Fingerhut, J. E. (2022) · Perspectives on Behavior Science · doi:10.1007/s40614-021-00282-2 · Cited by 53
“…a multilevel model can be made in relation to the measurement characteristics of the outcome (Declercq et al., 2019), according to the specific SCED used (Moeyaert et al., 2014b; Shadish et al., 2013), and the linear or nonlinear data pattern expected or observed (Shadish, Zuur, & Sullivan, 2014b).”
BackgroundMethodInfluentialMcDonald, S., Vieira, R., & Johnston, D. W. (2020) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2019.1711096 · Cited by 43
Testing the Intervention Effect in Single-Case Experiments: A Monte Carlo Simulation Study
Heyvaert, M., Moeyaert, M., Verkempynck, P., Van den Noortgate, W., Vervloet, M., Ugille, M., & Onghena, P. (2017) · The Journal of Experimental Education · doi:10.1080/00220973.2015.1123667 · Cited by 42
“For instance, twolevel HLMs can be extended to account for trends (linear and nonlinear; Shadish, Kyse, & Rindskopf, 2013; Van den Noortgate & Onghena, 2003b), autocorrelation (Van den Noorgate & Onghena, 2003a), unequal within-phase variances (Baek & Ferron, 2013), external events (Moeyaert, Ugille, Ferron, Beretvas,…”
BackgroundMethodInfluential
Metadata
- Authors
- Shadish, W. R., Kyse, E. N., & Rindskopf, D. M.
- Venue
- Psychological Methods
- Published
- 2013
- Volume
- 18
- Issue
- 3
- Pages
- 385-405
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2139776040
- Cited by
- 150
- References
- 15
- FWCI
- 10.61
- Type
- journal article
- Language
- en
- Open access
- closed
- Funders
- Department of Education (R305D100046 and R305D100033)
- University of California
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.