Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 180 of the 180 references it cites were found in a registry or an index; none of the 134 sources its claims cite is checked for support yet.

Analyzing data from single-case alternating treatments designs.

Open access
Manolov, Rumen, Onghena, Patrick · 2018 · Psychological Methods

Abstract

Alternating treatments designs (ATDs) have received comparatively less attention than other single-case experimental designs in terms of data analysis, as most analytical proposals and illustrations have been made in the context of designs including phases with several consecutive measurements in the same condition. One of the specific features of ATDs is the rapid (and usually randomly determined) alternation of conditions, which requires adapting the analytical techniques. First, we review the methodologically desirable features of ATDs, as well as the characteristics of the published single-case research using an ATD, which are relevant for data analysis. Second, we review several existing options for ATD data analysis. Third, we propose 2 new procedures, suggested as alternatives improving some of the limitations of extant analytical techniques. Fourth, we illustrate the application of existing techniques and the new proposals in order to discuss their differences and similarities. We advocate for the use of the new proposals in ATDs, because they entail meaningful comparisons between the conditions without assumptions about the design or the data pattern. We provide R code for all computations and for the graphical representation of the comparisons involved. (PsycINFO Database Record

Overview

CoChat Summary

TL;DR

The paper reviews existing methods for analyzing single-case alternating treatments designs (ATDs), reports a review of 47 published ATD studies from 2010–2015, and proposes two new procedures—ADISO and ALIV—for comparing adjacent conditions without assuming a specific data pattern. It advocates using these procedures with randomization tests and provides R code and a Shiny app.

Key findings

  • In the review of 47 ATD studies, visual analysis was the most common technique (75%), followed by mean/mean difference (72%); PND was used four times and PND-W once. Randomization of condition order was present in more than 50% of studies; only 36% had equal numbers of measurements per condition; 46% had no baseline phase.
  • Existing quantifications have limitations: mean difference ignores trend; PND-W is best when conditions occur equally often and can leave data unused; piecewise regression requires a baseline and can be misleading with crossing/opposite trends; LOESS requires subjective smoothing choices and may not perform well with typical short ATD series (review average 6.60 measurements per condition, median 5.38).
  • ADISO compares adjacent conditions and averages weighted differences; for Figure 1F data, five differences (8.15, 6.75, 6.60, 24.95, 9.80) yielded ADISO = 11.07, and ADISO-O gave 100% vs 83.33% superiority depending on segmentation. ALIV compares actual and linearly interpolated values; for Figure 3 data, differences had SD 10.72 and standardized ALIV = 0.65.
  • Randomization tests are natural for randomized ATDs; examples yielded one-tailed p = .000244 for mean difference vs .000122 for ALIV (Figure 1C), and p = .0952 vs .1071 (Sil et al., 2013), showing test-statistic choice can affect conclusions.

Methods

  • Methodological review plus proposal. Bibliographic search in PsycINFO up to January 1, 2016, using the exact term “alternating treatments design” in any field, focused on 2010–2015; 47 studies met criteria for being ATDs. The review is described as a convenience sample of online journal articles, excluding book chapters and dissertations.
  • Coded design and data features: presence/absence of randomization, equal measurement occasions per condition, baseline phase, overlap, linear/nonlinear trend, and analytical techniques reported. Operational definitions included reading design sections for “random” and derivatives, counting data points per condition, inspecting graphs for baseline phases, and visual assessment of trend.
  • Proposed ADISO (average difference between successive observations) and ALIV (actual and linearly interpolated values), illustrated on real ATD datasets and compared with existing techniques; R code and a user-friendly webpage are provided, and randomization tests are applied to some examples.

Limitations

  • Stated: formal evaluation via simulation is needed; ADISO’s segmentation choice is a limitation and its standard error is unknown; software for applying ADISO as a randomization-test statistic across all possible segmentations is not yet developed; ALIV does not use the first and last values in the sequence for comparison; randomization tests require random assignment in the design.
  • Stated: the review is a convenience sample of online journal articles (not book chapters or dissertations), and trend assessment was visual/subjective rather than model-based.
  • Evident from text: the paper illustrates rather than formally evaluates the new procedures; several recommendations rely on examples and prior literature, and the review’s coding of randomization and trend may be limited by what published articles report.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1037/met0000133

Cites

180 of 180 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 9 Not valid · 93 Not checked · 199 Own results and interpretations

96 more claims

Create a free account to see all 301 claims and to verify the 72 references still open.

See all claims — free account
  • It may state two separate claims as one. 109
  • Probably not a substantial claim. 145

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Research works printed but not linked: 4

  1. Busk, P. L., & Serlin, R. C. (1992). Meta-analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research designs and analysis: New directions for psychology and education (pp. 187-212). Hillsdale, NJ: Lawrence Erlbaum.
  2. Guyatt, G., Jaeschke, R., & McGinn, T. (2002). PART 2B1: Therapy and validity. N-of-1 randomized controlled trials. In G. Guyatt, D. Rennie, M. O. Meade, & D. J. Cook (Eds.), Users' guides to the medical literature (pp. 275-290). New York, NY: McGraw-Hill.
  3. Kempthorne, O. (1979). Sampling inference, experimental inference and observation inference. Sankhyā: The Indian Journal of Statistics, Series B, 40, 115-145.
  4. Rubin, D. (2005). Causal inference using potential outcomes. Journal of the American Statistical Association, 100, 322-331.

Cited by

See all 70

Metadata

Authors
Manolov, R., & Onghena, P.
Affiliations
Universitat de Barcelona (ES)
Corresponding author
Onghena, Patrick
Venue
Psychological Methods
Published
2018-09
Volume
23
Issue
3
Pages
480-504
ISSN
  • 1082-989X (ISSN-L)
  • 1939-1463
OpenAlex
W2589634095
Cited by
70
References
180
Citations by year
2026: 12 · 2025: 6 · 2024: 6 · 2023: 3 · 2022: 10 · 2021: 12 · 2020: 9 · 2019: 7 · 2018: 4
FWCI
7.27
Type
journal article
Language
en
Open access
Licence
other-oa
Locations
  • Psychological Methods (publishedVersion)
  • Lirias (publishedVersion, other-oa)
  • PubMed (publishedVersion)
  • Dipòsit Digital de la Universitat de Barcelona (Universitat de Barcelona) (acceptedVersion)
  • RECERCAT (Consorci de Serveis Universitaris de Catalunya) (submittedVersion, public-domain)
Topics
  • Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords
alternating treatments design, single-case experimental design, data analysis, treatment effect estimation, R code
MeSH terms
Data Interpretation, Statistical, Humans, Psychology, Regression Analysis, Research Design, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.