Comparing Visual and Statistical Analysis in Single-Case Studies Using Published Studies
Open accessAbstract
Little is known about the extent to which interrupted time series analysis (ITSA) can be applied to short, single-case study designs and whether those applications produce results consistent with visual analysis (VA). This article examines the extent to which ITSA can be applied to single-case study designs and compares the results based on two methods: ITSA and VA, using papers published in the Journal of Applied Behavior Analysis in 2010. The study was made possible by the development of software called UnGraph®, which facilitates the recovery of raw data from the graphs. ITSA was successfully applied to 94% of the examined graphs with the number of observations ranging from 8 to 136. Moderate to high lag-1 autocorrelations (>.50) were found for 46% of the data series. Effect sizes similar to group-level Cohen's d were identified based on the tertile distribution. Effects ranging from 0.00 to 0.99 were classified as small, those ranging from 1.00 to 2.49 as medium, and large effect sizes were defined as 2.50 or greater. Comparison of the conclusions from VA and ITSA had a low level of agreement (Kappa =.14, accounting for the agreement expected by chance). The results demonstrate that ITSA can be broadly implemented in applied behavior analysis research. These two methods should be viewed as complementary and used concurrently.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There can be disagreement between raters on whether change is present in visual analysis.
“For example, there can be disagreement between raters on whether change is present; therefore, inter-rater agreement often low in visual analysis (Harrington & Velicer, 2015).”
From A pilot case series of a brief acceptance and commitment therapy (ACT)-based guided self-help intervention for improving quality of life and mood in muscle disorders (2017) · cites this paper as Harrington & Velicer, 2015
Inter-rater agreement is often low in visual analysis.
“therefore, inter-rater agreement often low in visual analysis (Harrington & Velicer, 2015).”
From A pilot case series of a brief acceptance and commitment therapy (ACT)-based guided self-help intervention for improving quality of life and mood in muscle disorders (2017) · cites this paper as Harrington & Velicer, 2015
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,588 · see citing works
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · see citing works
Single Case Experimental Designs: Strategies for Studying Behavior Change
Barlow, D. H., Nock, M. K., & Hersen, M. (1976) · Cited by 3,036 · see citing works
Single-Case Research Designs: Methods for Clinical and Applied Settings
Kazdin, A. E. (1982) · Cited by 2,852 · see citing works
Single-case research designs: Methods for clinical and applied settings
Hayes, S. C. (1983) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/0005-7916(83)90018-6 · Cited by 2,677 · see citing works
Molenaar, P. C. M. (2004) · Measurement: Interdisciplinary Research & Perspective · doi:10.1207/s15366359mea0204_1 · Cited by 1,943 · see citing works
Combining nonoverlap and trend for single-case research: Tau-U
Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011) · Behavior Therapy · doi:10.1016/j.beth.2010.08.006 · Cited by 1,468 · see citing works
Single case experimental designs: Strategieṡ for studying behavior change
Matson, J. L. (1985) · Applied Research in Mental Retardation · doi:10.1016/0270-3092(85)90012-8 · Cited by 1,193 · see citing works
The new person-specific paradigm in psychology
Molenaar, P. C. M., & Campbell, C. G. (2009) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2009.01619.x · Cited by 1,088 · see citing works
IV.—On Least Squares and Linear Combination of Observations
Aitken, A. C. (1936) · Proceedings of the Royal Society of Edinburgh · doi:10.1017/s0370164600014346 · Cited by 954 · see citing works
Effect Size in Single-Case Research: A Review of Nine Nonoverlap Techniques
Parker, R. I., Vannest, K. J., & Davis, J. L. (2011) · Behavior Modification · doi:10.1177/0145445511399147 · Cited by 841 · see citing works
Maydeu‐Olivares, A., & McArdle, J. J. (2005) · Psychology Press eBooks · doi:10.4324/9781410612977 · Cited by 824 · see citing works
Single-case experimental designs: a systematic review of published research and current standards
Smith, J. D. (2012) · Psychological Methods · doi:10.1037/a0029312 · Cited by 630 · cited 3× · see citing works
“This journal was selected because it is a leading journal on the topic used by applied researchers, and it strongly promotes the use of VA rather than quantitative analysis methods (Shadish & Sullivan, 2011; Smith, 2012).”
Design and Analysis of Time-Series Experiments.
Newbold, P., Glass, G. V., Willson, V. L., & Gottman, J. M. (1976) · Journal of the American Statistical Association · doi:10.2307/2285785 · Cited by 595 · see citing works
Kratochwill, T. R., & Levin, J. R. (2010) · Psychological Methods · doi:10.1037/a0017736 · Cited by 497 · see citing works
Huitema, B. E. (2011) · Wiley Series in Probability and Statistics · doi:10.1002/9781118067475 · Cited by 464 · see citing works
Characteristics of single-case designs used to assess intervention effects in 2008
Shadish, W. R., & Sullivan, K. J. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0111-y · Cited by 338 · see citing works
A standardized mean difference effect size for single case designs
Hedges, L. V., Pustejovsky, J. E., & Shadish, W. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1052 · Cited by 308 · see citing works
The Analysis and Presentation of Graphic Data
Parsonson, B. S., & Baer, D. M. (1978) · Elsevier eBooks · doi:10.1016/b978-0-12-425850-1.50009-0 · Cited by 306 · see citing works
“PERHAPS IT WOULD BE BETTER NOT TO KNOW EVERYTHING”1
Baer, D. M. (1977) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1977.10-167 · Cited by 279 · see citing works
Peter, C. C. S., Vollmer, T. R., & Sloman, K. N. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-47 · Cited by 266 · see citing works
INCONSISTENT VISUAL ANALYSES OF INTRASUBJECT DATA
DeProspero, A., & Cohen, S. (1979) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1979.12-573 · Cited by 260 · see citing works
Molenaar, P. C. M. (2008) · Developmental Psychobiology · doi:10.1002/dev.20262 · Cited by 237 · see citing works
Matyas, T. A., & Greenwood, K. M. (1990) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1990.23-341 · Cited by 222 · see citing works
Interrupted time-series analysis with brief single-subject data.
Crosbie, J. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.61.6.966 · Cited by 220 · see citing works
Multilevel Meta-analysis: a comparison with traditional meta-analytical procedures
Van Den Noortgate, W., & Onghena, P. (2003) · Educational and Psychological Measurement · doi:10.1177/0013164403251027 · Cited by 218 · see citing works
Brossart, D. F., Parker, R. I., Olson, E. A., & Mahadevan, L. (2006) · Behavior Modification · doi:10.1177/0145445503261167 · Cited by 217 · see citing works
Combining single-case experimental data using hierarchical linear models.
Noortgate, W. V. D., & Onghena, P. (2003) · School Psychology Quarterly · doi:10.1521/scpq.18.3.325.22577 · Cited by 195 · see citing works
A multilevel meta-analysis of single-subject experimental design studies
Van den Noortgate, W., & Onghena, P. (2008) · Evidence-Based Communication Assessment and Intervention · doi:10.1080/17489530802505362 · Cited by 192 · see citing works
Hierarchical linear models for the quantitative integration of effect sizes in single-case research
Noortgate, W. V. D., & Onghena, P. (2003) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195492 · Cited by 190 · see citing works
AN INVESTIGATION OF DIFFERENTIAL REINFORCEMENT OF ALTERNATIVE BEHAVIOR WITHOUT EXTINCTION
Athens, E. S., & Vollmer, T. R. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-569 · Cited by 190 · see citing works
TIME‐SERIES ANALYSIS IN OPERANT RESEARCH1
Jones, R. R., Vaught, R. S., & Weinrott, M. R. (1977) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1977.10-151 · Cited by 187 · see citing works
EFFECTS OF SERIAL DEPENDENCY ON THE AGREEMENT BETWEEN VISUAL AND STATISTICAL INFERENCE1
Jones, R. R., Weinrott, M. R., & Vaught, R. S. (1978) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1978.11-277 · Cited by 186 · see citing works
Huitema, B. E. (2011) · Cited by 183 · see citing works
Cross-sectional time-series experiments: Some suggested statistical analyses.
Simonton, D. K. (1977) · Psychological Bulletin · doi:10.1037/0033-2909.84.3.489 · Cited by 159 · see citing works
First‐order integer valued AR processes with zero inflated poisson innovations
Jazi, M. A., Jones, G., & Lai, C. (2012) · Journal of Time Series Analysis · doi:10.1111/j.1467-9892.2012.00809.x · Cited by 158 · see citing works
Maggin, D. M., Swaminathan, H., Rogers, H. J., O’Keeffe, B. V., Sugai, G. M., & Horner, R. H. (2011) · Journal of School Psychology · doi:10.1016/j.jsp.2011.03.004 · Cited by 138 · see citing works
EFFECTS OF VIDEO MODELING ON TREATMENT INTEGRITY OF BEHAVIORAL INTERVENTIONS
Reed, F. D. D., Codding, R. S., Catania, C. N., & Maguire, H. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-291 · Cited by 114 · see citing works
Choe, G. H. (2005) · Springer eBooks · doi:10.1007/b138894 · Cited by 97 · see citing works
Carbone, V. J., Sweeney‐Kerwin, E. J., Attanasio, V., & Kasper, T. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-705 · Cited by 92 · see citing works
Telemetric monitoring in the behavior sciences
Goodwin, M. S., Velicer, W. F., & Intille, S. S. (2008) · Behavior Research Methods · doi:10.3758/brm.40.1.328 · Cited by 90 · see citing works
Self-citationComparing N = 1 Effect Size Indices in Presence of Autocorrelation
Manolov, R., & Solanas, A. (2008) · Behavior Modification · doi:10.1177/0145445508318866 · Cited by 88 · see citing works
Using UnGraph to extract data from image files: Verification of reliability and validity
Shadish, W. R., Brasil, I. C. C., Illingworth, D. A., White, K. D., Galindo, R. J., Nagler, E. D., & Rindskopf, D. M. (2009) · Behavior Research Methods · doi:10.3758/brm.41.1.177 · Cited by 87 · see citing works
The Aggregation of single-case results using hierarchical linear models
Van den Noortgate, W., & Onghena, P. (2007) · The Behavior Analyst Today · doi:10.1037/h0100613 · Cited by 86 · see citing works
BEHAVIIORAL COACHING TO IMPROVE OFFENSIVE LINE PASS‐BLOCKING SKILLS OF HIGH SCHOOL FOOTBALL ATHLETES
Stokes, J. V., Luiselli, J. K., Reed, D. D., & Fleming, R. K. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-463 · Cited by 85 · see citing works
Mevers, J. L., Fisher, W. W., & Kelley, M. E. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-425 · Cited by 78 · see citing works
Visual inspection of single-subject data: an empirical analysis.
Ottenbacher, K. J. (1990) · PubMed · Cited by 78 · see citing works
Irrelevant autocorrelation in least-squares intervention models.
Huitema, B. E., & McKean, J. W. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.1.104 · Cited by 76 · see citing works
Raiff, B. R., & Dallery, J. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-487 · Cited by 75 · see citing works
A Comparison of Alternative Approaches to the Analysis of Interrupted Time-Series
Harrop, J. W., & Velicer, W. F. (1985) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2001_2 · Cited by 73 · see citing works
Self-citationASSESSMENT AND TREATMENT OF ELOPEMENT MAINTAINED BY ACCESS TO STEREOTYPY
Falcomata, T. S., Roane, H. S., Feeney, B. J., & Stephenson, K. M. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-513 · Cited by 70 · see citing works
DISCRIMINATED FUNCTIONAL COMMUNICATION: A PROCEDURAL EXTENSION OF FUNCTIONAL COMMUNICATION TRAINING
Kuhn, D. E., Chirighin, A. E., & Zelenka, K. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-249 · Cited by 69 · see citing works
The Reliability and Accuracy of Time Series Model Identification
Velicer, W. F., & Harrop, J. W. (1983) · Evaluation Review · doi:10.1177/0193841x8300700408 · Cited by 66 · see citing works
Self-citationToussaint, K. A., & Tiger, J. H. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-181 · Cited by 65 · see citing works
USING COMPLEX AUDITORY—VISUAL SAMPLES TO PRODUCE EMERGENT RELATIONS IN CHILDREN WITH AUTISM
Groskreutz, N. C., Karsina, A., Miguel, C. F., & Groskreutz, M. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-131 · Cited by 64 · see citing works
When the Truth Hits You Between the Eyes
Bulté, I., & Onghena, P. (2011) · Methodology · doi:10.1027/1614-2241/a000042 · Cited by 55 · see citing works
Ülke-Kürkçüoğlu, B., & Kırcaali-İftar, G. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-717 · Cited by 52 · see citing works
ON THE RELATION BETWEEN REINFORCER EFFICACY AND PREFERENCE
Lee, M. S. H., Yu, C. T., Martin, T. L., & Martin, G. L. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-95 · Cited by 52 · see citing works
A SIMPLE AND POWERFUL TEST FOR AUTOCORRELATED ERRORS IN OLS INTERVENTION MODELS
Huitema, B. E., & McKean, J. W. (2000) · Psychological Reports · doi:10.2466/pr0.2000.87.1.3 · Cited by 48 · see citing works
FUNCTIONAL ANALYSIS AND TREATMENT OF AGGRESSION MAINTAINED BY PREFERRED CONVERSATIONAL TOPICS
Roscoe, E. M., Kindle, A. E., & Pence, S. T. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-723 · Cited by 44 · see citing works
Times Series Analysis Without Model Identification
Velicer, W. F., & McDonald, R. P. (1984) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr1901_2 · Cited by 42 · see citing works
Self-citationCarter, S. L. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-543 · Cited by 38 · see citing works
ANALYSIS OF DATA IN IDIOGRAPHIC RESEARCH
Ottenbacher, K. J. (1992) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/00002060-199208000-00002 · Cited by 38 · see citing works
Smith, J. D., Borckardt, J. J., & Nash, M. R. (2011) · Behavior Therapy · doi:10.1016/j.beth.2011.10.001 · Cited by 35 · see citing works
A BEHAVIORAL INTERVENTION FOR TEACHING TACKLING SKILLS TO HIGH SCHOOL FOOTBALL ATHLETES
Stokes, J. V., Luiselli, J. K., & Reed, D. D. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-509 · Cited by 33 · see citing works
Dolezal, D. N., & Kurtz, P. F. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-309 · Cited by 33 · see citing works
USE OF PEER‐MEDIATED INTERVENTION IN CHILDREN WITH ATTENTION DEFICIT HYPERACTIVITY DISORDER
MacAleese, A. N., & Wallace, M. D. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-547 · Cited by 31 · see citing works
Molenaar, P. C. M. (2008) · Sage eBooks · doi:10.4135/9781412976589.n5 · Cited by 29 · see citing works
Wilder, D. A., Allison, J., Nicholson, K., Abellon, O. E., & Saulnier, R. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-601 · Cited by 28 · see citing works
Wu, J. (2012) · Chapman & Hall/CRC eBooks · doi:10.1201/b11639-10 · Cited by 27 · see citing works
AN EVALUATION OF ADVANCE NOTICE TO INCREASE COMPLIANCE AMONG PRESCHOOLERS
Wilder, D. A., Nicholson, K., & Allison, J. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-751 · Cited by 26 · see citing works
FURTHER EXAMINATION OF DISCRIMINATED FUNCTIONAL COMMUNICATION
León, Y., Hausman, N. L., Kahng, S., & Becraft, J. L. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-525 · Cited by 25 · see citing works
The Effect of Autocorrelation on the Results of Visually Analyzing Data From Single-Subject Designs
Bengali, M. K., & Ottenbacher, K. J. (1998) · American Journal of Occupational Therapy · doi:10.5014/ajot.52.8.650 · Cited by 24 · see citing works
INCREASING SEAT BELT USE IN SERVICE VEHICLE DRIVERS WITH A GEARSHIFT DELAY
Houten, R. V., Malenfant, J. E. L., Reagan, I. J., Sifrit, K. J., Compton, R., & Tenenbaum, J. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-369 · Cited by 22 · see citing works
Travis, R. W., & Sturmey, P. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-745 · Cited by 21 · see citing works
Time-Series Intervention Analysis Using ITSACORR: Fatal Flaws
Huitema, B. E., McKean, J. W., & Laraway, S. (2007) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1193889780 · Cited by 18 · see citing works
AN EXPERIMENTAL ANALYSIS OF NEGATIVE REINFORCEMENT CONTINGENCIES FOR ADULT‐DELIVERED REPRIMANDS
Miller, J. R., Lerman, D. C., & Fritz, J. N. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-769 · Cited by 18 · see citing works
THE EFFECTS OF FIXED‐TIME ESCAPE ON INAPPROPRIATE AND APPROPRIATE CLASSROOM BEHAVIOR
Waller, R. D., & Higbee, T. S. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-149 · Cited by 17 · see citing works
Computer Programs for Interrupted Time Series Analysis: II A Quantitative Evaluation
Harrop, J. W., & Velicer, W. F. (1990) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2502_13 · Cited by 15 · see citing works
Self-citationComputer Programs for Interrupted Time Series Analysis: I. A Qualitative Evaluation
Harrop, J. W., & Velicer, W. F. (1990) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2502_12 · Cited by 6 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 4
- Aitken AC On least squares and lineal combination of observations Proceedings of the Royal Society of Edinburg H 1934 55 42 47
- Biosoft UnGraph for Windows (Version 5.0) 2004 Cambridge, U.K Author
- Harrop JW Velicer WF A comparison of three alternative methods of time series model identification Multivariate Behavioral Research 1985 20 27 44 26776274
- Van den Noortgate W Onghena P Combining singlecase experimental studies using hierarchical linear models School Psychology Quarterly 2003a 18 325 346
Cited by
Sorted by citations
Ewusie, J. E., Soobiah, C., Blondal, E., Beyene, J., Thabane, L., & Hamid, J. S. (2020) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s241085 · Cited by 155 · see citing works
Recommendations for Choosing Single-Case Data Analytical Techniques.
Manolov, R., & Moeyaert, M. (2017) · The Behavior Therapist · doi:10.1016/j.beth.2016.04.008 · Cited by 122 · see citing works
How Can Single-Case Data Be Analyzed? Software Resources, Tutorial, and Reflections on Analysis
Manolov, R., & Moeyaert, M. (2016) · Behavior Modification · doi:10.1177/0145445516664307 · Cited by 78 · see citing works
Randomized single-case AB phase designs: Prospects and pitfalls
Michiels, B., & Onghena, P. (2019) · Behavior Research Methods · doi:10.3758/s13428-018-1084-x · Cited by 77 · see citing works
Violin plots as visual tools in the meta-analysis of Single-Case Experimental Designs
Tanious, R., & Manolov, R. (2022) · Methodology · doi:10.5964/meth.9209 · Cited by 58 · see citing works
Maggin, D. M., Barton, E. E., Reichow, B., Lane, K. L., & Shogren, K. A. (2021) · Remedial and Special Education · doi:10.1177/07419325211051317 · Cited by 58 · see citing works
Analyzing data from single-case alternating treatments designs.
Manolov, R., & Onghena, P. (2017) · Psychological Methods · doi:10.1037/met0000133 · Cited by 57 · see citing works
Manolov, R., Losada, J., Chacón-Moscoso, S., & Sanduvete-Chaves, S. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00032 · Cited by 53 · see citing works
Power of a randomization test in a single case multiple baseline AB design
Bouwmeester, S., & Jongerling, J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0228355 · Cited by 55 · see citing works
Results reporting in single case experiments and single case meta-analysis
Vannest, K. J., Peltier, C., & Haas, A. (2018) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2018.04.029 · Cited by 51 · see citing works
A Visual Aid and Objective Rule Encompassing the Data Features of Visual Analysis
Manolov, R., & Vannest, K. J. (2019) · Behavior Modification · doi:10.1177/0145445519854323 · Cited by 50 · see citing works
Single-case synthesis tools II: Comparing quantitative outcome measures
Zimmerman, K. N., Pustejovsky, J. E., Ledford, J. R., Barton, E. E., Severini, K. E., & Lloyd, B. P. (2018) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2018.02.001 · Cited by 46 · see citing works
Smith, J. D., Eichler, W. C., Norman, K. R., & Smith, S. R. (2014) · Journal of Personality Assessment · doi:10.1080/00223891.2014.955917 · Cited by 43 · see citing works
Autocorrelation and estimates of treatment effect size for single‐case experimental design data
Barnard‐Brak, L., Watkins, L., & Richman, D. M. (2021) · Behavioral Interventions · doi:10.1002/bin.1783 · Cited by 41 · see citing works
Confidence intervals for single-case effect size measures based on randomization test inversion
Michiels, B., Heyvaert, M., Meulders, A., & Onghena, P. (2016) · Behavior Research Methods · doi:10.3758/s13428-016-0714-4 · Cited by 40 · see citing works
One by One: Accumulating Evidence by using Meta-Analytical Procedures for Single-Case Experiments
Onghena, P., Michiels, B., Jamshidi, L., Moeyaert, M., & Noortgate, W. V. D. (2017) · Brain Impairment · doi:10.1017/brimp.2017.25 · Cited by 40 · see citing works
Barton, E. E., Meadan, H., & Fettig, A. (2018) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2018.11.001 · Cited by 36 · see citing works
Efficacy of metacognitive therapy in improving mental health: A meta‐analysis of single‐case studies
Rochat, L., Manolov, R., & Billieux, J. (2017) · Journal of Clinical Psychology · doi:10.1002/jclp.22567 · Cited by 34 · see citing works
Lavoie, M., Bier, N., Laforce, R., & Macoir, J. (2020) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2019.1570943 · Cited by 30 · see citing works
A primer for using multilevel models to meta‐analyze single case design data with AB phases
Becraft, J. L., Borrero, J. C., Sun, S., & McKenzie, A. A. (2020) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.698 · Cited by 26 · see citing works
Dealing with missing data by EM in single-case studies
Chen, L., Feng, Y., Wu, P.-J., & Peng, C. J. (2019) · Behavior Research Methods · doi:10.3758/s13428-019-01210-8 · Cited by 27 · see citing works
Somer, E., Gische, C., & Miočević, M. (2022) · Evaluation & the Health Professions · doi:10.1177/01632787211071136 · Cited by 16 · see citing works
Assessing Nonoverlap in Single-Case Data: Strengths, Challenges, and Recommendations
Manolov, R., & Tanious, R. (2024) · Journal of Behavioral Education · doi:10.1007/s10864-024-09552-w · Cited by 24 · see citing works
Assessing Consistency in Single-Case A-B-A-B Phase Designs
Tanious, R., De, T. K., Michiels, B., den Noortgate, W. V., & Onghena, P. (2020) · Behavior Modification · doi:10.1177/0145445519837726 · Cited by 22 · see citing works
Extrapolating baseline trend in single-case data: Problems and tentative solutions
Manolov, R., Solanas, A., & Sierra, V. (2018) · Behavior Research Methods · doi:10.3758/s13428-018-1165-x · Cited by 23 · see citing works
Single-Case Experimental Designs: Clinical Research and Practice
Vlaeyen, J. W. S., Onghena, P., Vannest, K. J., & Kratochwill, T. R. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-818697-8.00191-6 · Cited by 22 · see citing works
Barnard-Brak, L. M., Richman, D. M., Little, T. D., & Yang, Z. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2017.12.003 · Cited by 20 · see citing works
Houtveen, J. H., van der Sluijs, J. V. E., Thorsell, S., Kloth, S. A. M., & Geenen, R. (2022) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2022.110724 · Cited by 19 · see citing works
Batley, P. N., Mehta, S. S., & Hitchcock, J. H. (2020) · Behavioral Disorders · doi:10.1177/0198742920930704 · Cited by 19 · see citing works
Aydın, O., & Tanious, R. (2022) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.928 · Cited by 18 · see citing works
Questionable Research Practices in Single-Case Experimental Designs: Examples and Possible Solutions
Tincani, M., & Travers, J. C. (2022) · Springer eBooks · doi:10.1007/978-3-031-04968-2_12 · Cited by 18 · see citing works
Li, H., Luo, W., Baek, E., Thompson, C. G., & Lam, K. H. (2023) · Psychological Methods · doi:10.1037/met0000607 · Cited by 18 · see citing works
Batley, P. N., & Hedges, L. V. (2021) · Behavior Research Methods · doi:10.3758/s13428-020-01522-0 · Cited by 17 · see citing works
Intervenciones para mejorar el juego de los niños con autismo en el patio de recreo
Gómez, A. G., Ambrosio-Bravo, M., & Gil-Diaz, L. (2020) · Estudios sobre Educación · doi:10.15581/004.38.253-278 · Cited by 15 · see citing works
Fedewa, M. P., Watkins, L., Barnard-Brak, L. M., & Akemoğlu, Y. (2022) · Review Journal of Autism and Developmental Disorders · doi:10.1007/s40489-022-00343-5 · Cited by 14 · see citing works
Effects of Rational Emotive Behavior Therapy on Academic Grit
Sari, S. A., Sugara, G. S., & Isti‘adah, F. N. (2022) · Journal of Rational-Emotive & Cognitive-Behavior Therapy · doi:10.1007/s10942-022-00444-4 · Cited by 14 · see citing works
Books and Toddlers in Child Care: Under What Conditions are Children Most Engaged?
Gardner‐Neblett, N., Holochwost, S. J., Gallagher, K. C., Iruka, I. U., Odom, S. L., & Bruno, E. P. (2017) · Child & Youth Care Forum · doi:10.1007/s10566-017-9391-4 · Cited by 14 · see citing works
A Simulation Study on Two Analytical Techniques for Alternating Treatments Designs
Manolov, R. (2018) · Behavior Modification · doi:10.1177/0145445518777875 · Cited by 13 · see citing works
Borst, M., Moeyaert, M., & van Rood, Y. (2024) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2024.2314883 · Cited by 12 · see citing works
Russ, K. A., Holochwost, S. J., Perkins, S. M., Stegenga, K. A., Jacob, S. A., Delgado, D. C., Henley, A. K., Haase, J. E., & Robb, S. L. (2020) · The Journal of Alternative and Complementary Medicine · doi:10.1089/acm.2019.0413 · Cited by 12 · see citing works
Tutorial: Artificial neural networks to analyze single-case experimental designs.
Lanovaz, M. J., & Bailey, J. D. (2022) · Psychological Methods · doi:10.1037/met0000487 · Cited by 11 · see citing works
Misjudgment of interrupted time-series graphs due to serial dependence: Replication of
Bishara, A. J., Peller, J., & Galuska, C. M. (2021) · Judgment and Decision Making · doi:10.1017/s1930297500007786 · Cited by 11 · see citing works
Vroegindeweij, A., Nijhof, L. N., Onghena, P., van de Putte, E. M., Nijhof, S. L., & Houtveen, J. H. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02167-5 · Cited by 11 · see citing works
A proposal for the assessment of replication of effects in single‐case experimental designs
Manolov, R., Tanious, R., & Fernández‐Castilla, B. (2022) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.923 · Cited by 11 · see citing works
Maitland, D. W. M., & Lewis, J. A. (2022) · The Psychological Record · doi:10.1007/s40732-022-00526-w · Cited by 9 · see citing works
Developing a change approach for the transition to a high performance organization
de Waal, A., & Heijtel, I. (2017) · Measuring Business Excellence · doi:10.1108/mbe-03-2016-0015 · Cited by 11 · see citing works
Wallin, E., Parling, T., Weineland, S., & Dahl, J. C. (2018) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2018.08.007 · Cited by 11 · see citing works
A Clarification of Slope and Scale
Kinney, C. E. L. (2020) · Behavior Modification · doi:10.1177/0145445520953366 · Cited by 11 · see citing works
Assessing Consistency in Single-Case Data Features Using Modified Brinley Plots
Manolov, R., & Tanious, R. (2020) · Behavior Modification · doi:10.1177/0145445520982969 · Cited by 11 · see citing works
Correspondence between Fail-Safe k and Dual-Criteria Methods: Analysis of Data Series Stability
Falligant, J. M., Kranak, M. P., Schmidt, J. D., & Rooker, G. W. (2020) · Perspectives on Behavior Science · doi:10.1007/s40614-020-00255-x · Cited by 11 · see citing works
Barnard-Brak, L., Richman, D., & Watkins, L. (2020) · Perspectives on Behavior Science · doi:10.1007/s40614-020-00258-8 · Cited by 11 · see citing works
Bayesian Time‐Series Models in Single Case Experimental Designs: A Tutorial for Trauma Researchers
Batley, P. N., Contractor, A. A., & Caldas, S. V. (2020) · Journal of Traumatic Stress · doi:10.1002/jts.22614 · Cited by 10 · see citing works
Graham, C. D., Chalder, T., Rose, M. R., Gavriloff, D., McCracken, L. M., & Weinman, J. (2017) · The Cognitive Behaviour Therapist · doi:10.1017/s1754470x17000022 · Cited by 9 · see citing works
Taylor, J., Allen, L. M., Van, J., & Moohr, M. L. (2024) · Journal of Emotional and Behavioral Disorders · doi:10.1177/10634266241235933 · Cited by 9 · see citing works
Zhang, D., Johnstone, S. J., Li, H., Luo, X., & Sun, L. (2021) · Behaviour Change · doi:10.1017/bec.2021.26 · Cited by 9 · see citing works
Manolov, R. (2017) · Anuario de Psicología/The UB Journal of Psychology · doi:10.1016/j.anpsic.2017.05.004 · Cited by 9 · see citing works
Evaluating High Performance the Evidence-Based Way: The Case of the Swagelok Transformers
de Waal, A. (2017) · SAGE Open · doi:10.1177/2158244017736801 · Cited by 9 · see citing works
Consistency in Single-Case ABAB Phase Designs: A Systematic Review
Tanious, R., De, T. K., Michiels, B., Noortgate, W. V. D., & Onghena, P. (2019) · Behavior Modification · doi:10.1177/0145445519853793 · Cited by 9 · see citing works
Noortgate, W. V. D., & Onghena, P. (2024) · Psychologica Belgica · doi:10.5334/pb.1307 · Cited by 8 · see citing works
Zettle, R. D. (2020) · The Psychological Record · doi:10.1007/s40732-020-00394-2 · Cited by 8 · see citing works
Why and How to Deal With Diurnal Cyclic Patterns in Ambulatory Assessment of Emotions
van de Maat, R., Lataster, J., & Verboon, P. (2020) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000579 · Cited by 8 · see citing works
Treatment for social media addiction with hypnotic-oriented counseling
Sugara, G. S., & Fadhilah, R. (2024) · Discover Psychology · doi:10.1007/s44202-024-00125-6 · Cited by 7 · see citing works
Li, H. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02464-7 · Cited by 7 · see citing works
Considerations in Writing About Single-Case Experimental Design Studies
Skolasky, R. L. (2016) · Cognitive and Behavioral Neurology · doi:10.1097/wnn.0000000000000112 · Cited by 7 · see citing works
Sulu, M. D., Martella, R. C., Aydın, O., Bolshokova, V. L., & Erden, E. (2023) · Journal of Developmental and Physical Disabilities · doi:10.1007/s10882-023-09890-z · Cited by 7 · see citing works
Intensive and non-intensive treatment of lexical anomia are equally efficient in post-stroke aphasia
Monetta, L., Lavoie, M., Routhier, S., & Macoir, J. (2020) · Neurocase · doi:10.1080/13554794.2020.1868534 · Cited by 7 · see citing works
Falligant, J. M., Kranak, M. P., & Hagopian, L. P. (2021) · Perspectives on Behavior Science · doi:10.1007/s40614-021-00313-y · Cited by 7 · see citing works
Thurmann-Moe, A. C., Melby‐Lervåg, M., & Lervåg, A. (2021) · Annals of Dyslexia · doi:10.1007/s11881-021-00225-1 · Cited by 6 · see citing works
Smartphone use as an efficient tool to improve anomia in primary progressive aphasia
Joubert, S., Maquestiaux, F., Enriquez-Rosas, A., Villalpando, J., Brodeur, C., & Bier, N. (2023) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2023.2181824 · Cited by 4 · see citing works
Dayo, N., Said‐Metwaly, S., & Noortgate, W. V. D. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13468 · Cited by 6 · see citing works
Fallon, L. M., Collier‐Meek, M. A., Famolare, G. M., DeFouw, E. R., & Gould, K. M. (2021) · School Psychology Review · doi:10.1080/2372966x.2020.1853485 · Cited by 6 · see citing works
Single Case Experimental Design: A New Approach for Non-invasive Brain Stimulation Research?
Pate, J. W., & McCambridge, A. B. (2021) · Frontiers in Neuroergonomics · doi:10.3389/fnrgo.2021.678579 · Cited by 6 · see citing works
Kranak, M. P., & Hall, S. S. (2021) · Perspectives on Behavior Science · doi:10.1007/s40614-021-00290-2 · Cited by 6 · see citing works
Ewusie, J. E., Beyene, J., Thabane, L., Straus, S. E., & Hamid, J. S. (2021) · The International Journal of Biostatistics · doi:10.1515/ijb-2020-0046 · Cited by 6 · see citing works
Analyzing data in single-case experimental designs: Objectives and available software options
Manolov, R., & Rochat, L. (2024) · Journal of Behavioral and Cognitive Therapy · doi:10.1016/j.jbct.2024.100511 · Cited by 5 · see citing works
Robert, C. (2019) · L’Année psychologique · doi:10.3917/anpsy1.191.0055 · Cited by 5 · see citing works
Kinney, C. E. L., Weatherly, N. L., Burns, G. N., & Nicholson, K. (2022) · The Psychological Record · doi:10.1007/s40732-022-00522-0 · Cited by 5 · see citing works
Richman, D. M., Barnard-Brak, L. M., & Watkins, L. (2022) · Journal of Intellectual Disability Research · doi:10.1111/jir.12966 · Cited by 5 · see citing works
Effects on Wellbeing of Exposure to Dog Videos Before a Stressor
Ein, N., Gervasio, J., Reed, M. J., & Vickers, K. (2022) · Anthrozoös · doi:10.1080/08927936.2022.2149925 · Cited by 5 · see citing works
Assessing Consistency in Single-Case Alternation Designs
Manolov, R., Tanious, R., De, T. K., & Onghena, P. (2020) · Behavior Modification · doi:10.1177/0145445520923990 · Cited by 5 · see citing works
Manolov, R., & Onghena, P. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02230-1 · Cited by 5 · see citing works
Li, H., Baek, E., Luo, W., Du, W., & Lam, K. H. (2024) · Evaluation & the Health Professions · doi:10.1177/01632787241259500 · Cited by 4 · see citing works
Maydew, H., & Atkinson, S. (2022) · Educational Psychology in Practice · doi:10.1080/02667363.2022.2033957 · Cited by 7 · see citing works
A Comparison of Single-Case Effect Measures Using Check-In Check-Out Data
Peart, A. M., Drevon, D. D., & Jasper, A. D. (2024) · Behavior Modification · doi:10.1177/01454455241233738 · Cited by 4 · see citing works
Drevon, D. D., Peart, A. M., & Koval, E. T. (2023) · School Psychology Review · doi:10.1080/2372966x.2023.2273822 · Cited by 3 · see citing works
A Meta-Visual-Analysis of Single-Case Experimental Design Research
Kinney, C. E. L., Dowdy, A., & Wolfe, K. (2025) · Behavior Modification · doi:10.1177/01454455251320686 · Cited by 3 · see citing works
McInerney, M., Imms, C., Carding, P. N., & Reddihough, D. S. (2021) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2021.100872 · Cited by 3 · see citing works
Amado-Fuentes, M., Delgado, M. G., Gómez, A. G., & My works are correct, B. M. I. A. A. I. S. R. L. M. T. I. D. A.-M. (2021) · Children · doi:10.3390/children8111073 · Cited by 4 · see citing works
Sample Size in Behavioral Research: A Systematic Review of JEAB and JABA from 2009 to 2018
Schrimp, A. J., Griffith, J. D., Potoczak, K., Hatvany, T. C., Norwood, A. E. Q., & Conley, A. A. (2022) · Revista Brasileira de Análise do Comportamento · doi:10.18542/rebac.v18i2.13634 · Cited by 3 · see citing works
Chin, Y. C., Luna, O., & Witts, B. N. (2024) · Behavior Analysis in Practice · doi:10.1007/s40617-024-00970-w · Cited by 3 · see citing works
Dialectical behavior therapy in the treatment of perfectionism: A Single case design
Sugara, G. S., Puspitawati, A., Arumsari, C., Rahimsyah, A. P., & Muhajirin, M. (2023) · Konselor · doi:10.24036/0202312246-0-86 · Cited by 3 · see citing works
Single-case design and evaluation in R: An introduction and tutorial for school psychologists
McGill, R. J. (2016) · International Journal of School & Educational Psychology · doi:10.1080/21683603.2016.1173610 · Cited by 3 · see citing works
Laureano, B., & Falligant, J. M. (2022) · Behavior Modification · doi:10.1177/01454455221099648 · Cited by 3 · see citing works
Vives-Vilarroig, J., Ruíz-Bernardo, P., & Gómez, A. G. (2024) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-023-06174-5 · Cited by 2 · see citing works
Stutvoet, M. D., Vroegindeweij, A., Houtveen, J. H., van Litsenburg, R. R. L., van de Putte, E. M., Veltkamp, R. C., & Nijhof, S. L. (2025) · Trials · doi:10.1186/s13063-025-08960-1 · Cited by 2 · see citing works
Xue, Y., Moeyaert, M., Baek, E., Solomon, B. G., & Noortgate, W. V. D. (2025) · Journal of Behavioral Education · doi:10.1007/s10864-025-09603-w · Cited by 2 · see citing works
Effectiveness of acceptance and commitment therapy among adolescent athletes: a systematic review
Ptáček, M., Lugo, R. G., & Jelínek, M. (2026) · International Journal of Sport and Exercise Psychology · doi:10.1080/1612197x.2026.2645207 · Cited by 3 · see citing works
Comeau, N., Monetta, L., & Schneider, C. (2022) · Neurocase · doi:10.1080/13554794.2022.2068374 · Cited by 2 · see citing works
Proposals for Incorporating the Reversal in Single-Case ABAB Designs Data Analysis
Tanious, R., & Manolov, R. (2024) · The Journal of Experimental Education · doi:10.1080/00220973.2024.2382485 · Cited by 1 · see citing works
Hassan, E. T., Gracey, F., & Blake, J. (2025) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465825101124 · Cited by 1 · see citing works
- Authors
- Harrington, M., & Velicer, W. F.
- Affiliations
- University of Rhode Island (US)
- Corresponding author
- Velicer, Wayne F.
- ORCID
- Velicer, Wayne F.
- Venue
- Multivariate Behavioral Research
- Publisher
- Informa UK Limited
- Published
- 2015-03-04
- Volume
- 50
- Issue
- 2
- Pages
- 162-183
- ISSN
- 0027-3171 (ISSN-L)
- 1532-7906
- PMC
- PMC4677800
- OpenAlex
- W1969454087
- Cited by
- 114
- References
- 80
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 14 · 2023: 9 · 2022: 18 · 2021: 18 · 2020: 16 · 2019: 6 · 2018: 7 · 2017: 10 · 2016: 6 · 2014: 1
- FWCI
- 19.39
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Multivariate Behavioral Research (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Journal of Media Literacy Education (submittedVersion, cc-by)
- Funders
- National Institute on Drug Abuse (R01DA020112, DA020112)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- interrupted time series, visual analysis, single-case experimental design, autocorrelation, effect size, applied behavior analysis, measurement agreement
- MeSH terms
- Humans, Research Design, Reproducibility of Results, Behavioral Research, Interrupted Time Series Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review