The DOI matches its Crossref record; 180 of the 180 references it cites were found in a registry or an index; none of the 134 sources its claims cite is checked for support yet.
Analyzing data from single-case alternating treatments designs.
Open accessAbstract
Alternating treatments designs (ATDs) have received comparatively less attention than other single-case experimental designs in terms of data analysis, as most analytical proposals and illustrations have been made in the context of designs including phases with several consecutive measurements in the same condition. One of the specific features of ATDs is the rapid (and usually randomly determined) alternation of conditions, which requires adapting the analytical techniques. First, we review the methodologically desirable features of ATDs, as well as the characteristics of the published single-case research using an ATD, which are relevant for data analysis. Second, we review several existing options for ATD data analysis. Third, we propose 2 new procedures, suggested as alternatives improving some of the limitations of extant analytical techniques. Fourth, we illustrate the application of existing techniques and the new proposals in order to discuss their differences and similarities. We advocate for the use of the new proposals in ATDs, because they entail meaningful comparisons between the conditions without assumptions about the design or the data pattern. We provide R code for all computations and for the graphical representation of the comparisons involved. (PsycINFO Database Record
Overview
CoChat Summary
TL;DR
The paper reviews existing methods for analyzing single-case alternating treatments designs (ATDs), reports a review of 47 published ATD studies from 2010–2015, and proposes two new procedures—ADISO and ALIV—for comparing adjacent conditions without assuming a specific data pattern. It advocates using these procedures with randomization tests and provides R code and a Shiny app.
Key findings
- In the review of 47 ATD studies, visual analysis was the most common technique (75%), followed by mean/mean difference (72%); PND was used four times and PND-W once. Randomization of condition order was present in more than 50% of studies; only 36% had equal numbers of measurements per condition; 46% had no baseline phase.
- Existing quantifications have limitations: mean difference ignores trend; PND-W is best when conditions occur equally often and can leave data unused; piecewise regression requires a baseline and can be misleading with crossing/opposite trends; LOESS requires subjective smoothing choices and may not perform well with typical short ATD series (review average 6.60 measurements per condition, median 5.38).
- ADISO compares adjacent conditions and averages weighted differences; for Figure 1F data, five differences (8.15, 6.75, 6.60, 24.95, 9.80) yielded ADISO = 11.07, and ADISO-O gave 100% vs 83.33% superiority depending on segmentation. ALIV compares actual and linearly interpolated values; for Figure 3 data, differences had SD 10.72 and standardized ALIV = 0.65.
- Randomization tests are natural for randomized ATDs; examples yielded one-tailed p = .000244 for mean difference vs .000122 for ALIV (Figure 1C), and p = .0952 vs .1071 (Sil et al., 2013), showing test-statistic choice can affect conclusions.
Methods
- Methodological review plus proposal. Bibliographic search in PsycINFO up to January 1, 2016, using the exact term “alternating treatments design” in any field, focused on 2010–2015; 47 studies met criteria for being ATDs. The review is described as a convenience sample of online journal articles, excluding book chapters and dissertations.
- Coded design and data features: presence/absence of randomization, equal measurement occasions per condition, baseline phase, overlap, linear/nonlinear trend, and analytical techniques reported. Operational definitions included reading design sections for “random” and derivatives, counting data points per condition, inspecting graphs for baseline phases, and visual assessment of trend.
- Proposed ADISO (average difference between successive observations) and ALIV (actual and linearly interpolated values), illustrated on real ATD datasets and compared with existing techniques; R code and a user-friendly webpage are provided, and randomization tests are applied to some examples.
Limitations
- Stated: formal evaluation via simulation is needed; ADISO’s segmentation choice is a limitation and its standard error is unknown; software for applying ADISO as a randomization-test statistic across all possible segmentations is not yet developed; ALIV does not use the first and last values in the sequence for comparison; randomization tests require random assignment in the design.
- Stated: the review is a convenience sample of online journal articles (not book chapters or dissertations), and trend assessment was visual/subjective rather than model-based.
- Evident from text: the paper illustrates rather than formally evaluates the new procedures; several recommendations rely on examples and prior literature, and the review’s coding of randomization and trend may be limited by what published articles report.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 9 Not valid · 93 Not checked · 199 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is also common to accompany this quantification by some measure of dispersion (usually range): this was the case in 70% of the studies reporting means.”
§ Analyzing data from single-case alternating treatments designs > Existing Analytical Techniques: Mean Difference.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“another relevant data feature in ATDs is that the number of measurements per condition is not always equal, as suggested by our review.”
§ Analyzing data from single-case alternating treatments designs > How can ATD data be analyzed? Analytical Techniques Reviewed and New Proposals
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the most commonly used analytical strategy in the published research reviewed was visual analysis”
§ Analyzing data from single-case alternating treatments designs > How can ATD data be analyzed? Analytical Techniques Reviewed and New Proposals
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The segmentation chosen for ADISO provides slightly larger values than ALIV and the regression-Running head: ATD DATA ANALYSIS based models”
§ Analyzing data from single-case alternating treatments designs > Running head: ATD DATA ANALYSIS
plainstates an associationSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In terms of the percentage of comparisons for which one condition is superior to the other, the results of ADISO-O and PND-W are similar.”
§ Analyzing data from single-case alternating treatments designs > Applying the Analytical Options
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
96 more claims
Create a free account to see all 301 claims and to verify the 72 references still open.
- It may state two separate claims as one. 109
- Probably not a substantial claim. 145
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Analyzing data from single-case alternating treatments designs > Running head: ATD DATA ANALYSIS
“In terms of the application of a randomization test in the Eilers and Hayes (2013) study”
§ Analyzing data from single-case alternating treatments designs > References
“38. Eilers, H. J., & Hayes, S. C. (2015). Exposure and response prevention therapy with cognitive defusion exercises to reduce repetitive and restrictive behaviors displayed by children with autism spectrum disorder. Research in Autism Spectrum Disorders, 19, 18-31.”
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
(1984) · Cited by 81,983
Case Study Research: Design and Methods
Chaiklin, H. (1991) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-199102000-00025 · Cited by 39,535
Building Theories from Case Study Research
Eisenhardt, K. M. (1989) · The Academy of Management Review · doi:10.2307/258557 · Cited by 17,455
Five misunderstandings about case-study research
Flyvbjerg, B. (2006) · Qualitative Inquiry · doi:10.1177/1077800405284363 · Cited by 14,266
(2002) · Cited by 13,211
David, F. N., & Tukey, J. W. (1977) · Biometrics · doi:10.2307/2529486 · Cited by 12,708
Building theories from case study research
Eisenhardt, K. M. (2009) · STUDI ORGANIZZATIVI · doi:10.3280/so2008-002004 · Cited by 11,299
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436
(2000) · Cited by 9,198
Research works printed but not linked: 4
- Busk, P. L., & Serlin, R. C. (1992). Meta-analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research designs and analysis: New directions for psychology and education (pp. 187-212). Hillsdale, NJ: Lawrence Erlbaum.
- Guyatt, G., Jaeschke, R., & McGinn, T. (2002). PART 2B1: Therapy and validity. N-of-1 randomized controlled trials. In G. Guyatt, D. Rennie, M. O. Meade, & D. J. Cook (Eds.), Users' guides to the medical literature (pp. 275-290). New York, NY: McGraw-Hill.
- Kempthorne, O. (1979). Sampling inference, experimental inference and observation inference. Sankhyā: The Indian Journal of Statistics, Series B, 40, 115-145.
- Rubin, D. (2005). Causal inference using potential outcomes. Journal of the American Statistical Association, 100, 322-331.
Cited by
Mixed Methods Single Case Research: State of the Art and Future Directions
Onghena, P., Maes, B., & Heyvaert, M. (2018) · Journal of Mixed Methods Research · doi:10.1177/1558689818789530 · Cited by 138
Self-citationTanious, R., & Onghena, P. (2020) · Behavior Research Methods · doi:10.3758/s13428-020-01502-4 · Cited by 101
Self-citationRandomized single-case AB phase designs: Prospects and pitfalls
Michiels, B., & Onghena, P. (2019) · Behavior Research Methods · doi:10.3758/s13428-018-1084-x · Cited by 77
Self-citationRandomized Single-Case Experimental Designs in Healthcare Research: What, Why, and How?
Tanious, R., & Onghena, P. (2019) · Healthcare · doi:10.3390/healthcare7040143 · Cited by 63
Self-citationA Visual Aid and Objective Rule Encompassing the Data Features of Visual Analysis
Manolov, R., & Vannest, K. J. (2019) · Behavior Modification · doi:10.1177/0145445519854323 · Cited by 55
Self-citation
Metadata
- Authors
- Manolov, R., & Onghena, P.
- Affiliations
- Universitat de Barcelona (ES)
- Corresponding author
- Onghena, Patrick
- Venue
- Psychological Methods
- Published
- 2018-09
- Volume
- 23
- Issue
- 3
- Pages
- 480-504
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2589634095
- Cited by
- 70
- References
- 180
- Citations by year
- 2026: 12 · 2025: 6 · 2024: 6 · 2023: 3 · 2022: 10 · 2021: 12 · 2020: 9 · 2019: 7 · 2018: 4
- FWCI
- 7.27
- Type
- journal article
- Language
- en
- Open access
- green
- lirias.kuleuven.be
- Licence
- other-oa
- Locations
- Psychological Methods (publishedVersion)
- Lirias (publishedVersion, other-oa)
- PubMed (publishedVersion)
- Dipòsit Digital de la Universitat de Barcelona (Universitat de Barcelona) (acceptedVersion)
- RECERCAT (Consorci de Serveis Universitaris de Catalunya) (submittedVersion, public-domain)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- alternating treatments design, single-case experimental design, data analysis, treatment effect estimation, R code
- MeSH terms
- Data Interpretation, Statistical, Humans, Psychology, Regression Analysis, Research Design, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-18, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-18 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Analyzing data from single-case alternating treatments designs. https://app.cochat.ai/papers/10.1037/met0000133
- The paper
- Manolov, R., & Onghena, P. (2018). Analyzing data from single-case alternating treatments designs. Psychological Methods. https://doi.org/10.1037/met0000133
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.