The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
Comparing Visual and Statistical Analysis in Single-Case Studies Using Published Studies
Open accessAbstract
Little is known about the extent to which interrupted time series analysis (ITSA) can be applied to short, single-case study designs and whether those applications produce results consistent with visual analysis (VA). This article examines the extent to which ITSA can be applied to single-case study designs and compares the results based on two methods: ITSA and VA, using papers published in the Journal of Applied Behavior Analysis in 2010. The study was made possible by the development of software called UnGraph®, which facilitates the recovery of raw data from the graphs. ITSA was successfully applied to 94% of the examined graphs with the number of observations ranging from 8 to 136. Moderate to high lag-1 autocorrelations (>.50) were found for 46% of the data series. Effect sizes similar to group-level Cohen's d were identified based on the tertile distribution. Effects ranging from 0.00 to 0.99 were classified as small, those ranging from 1.00 to 2.49 as medium, and large effect sizes were defined as 2.50 or greater. Comparison of the conclusions from VA and ITSA had a low level of agreement (Kappa =.14, accounting for the agreement expected by chance). The results demonstrate that ITSA can be broadly implemented in applied behavior analysis research. These two methods should be viewed as complementary and used concurrently.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“For example, there can be disagreement between raters on whether change is present; therefore, inter-rater agreement often low in visual analysis (Harrington & Velicer, 2015).”
From A pilot case series of a brief acceptance and commitment therapy (ACT)-based guided self-help intervention for improving quality of life and mood in muscle disorders (2017) · cites this paper as Harrington & Velicer, 2015
The cited source says what the claim attributes to it.
In the citing paper
“therefore, inter-rater agreement often low in visual analysis (Harrington & Velicer, 2015).”
From A pilot case series of a brief acceptance and commitment therapy (ACT)-based guided self-help intervention for improving quality of life and mood in muscle disorders (2017) · cites this paper as Harrington & Velicer, 2015
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,603
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Single Case Experimental Designs: Strategies for Studying Behavior Change
Barlow, D. H., Nock, M. K., & Hersen, M. (1976) · Cited by 3,036
Single-Case Research Designs: Methods for Clinical and Applied Settings
Kazdin, A. E. (1982) · Cited by 2,852
Single-case research designs: Methods for clinical and applied settings
Hayes, S. C. (1983) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/0005-7916(83)90018-6 · Cited by 2,677
Molenaar, P. C. M. (2004) · Measurement: Interdisciplinary Research & Perspective · doi:10.1207/s15366359mea0204_1 · Cited by 1,943
Combining nonoverlap and trend for single-case research: Tau-U
Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011) · Behavior Therapy · doi:10.1016/j.beth.2010.08.006 · Cited by 1,468
Single case experimental designs: Strategieṡ for studying behavior change
Matson, J. L. (1985) · Applied Research in Mental Retardation · doi:10.1016/0270-3092(85)90012-8 · Cited by 1,191
The new person-specific paradigm in psychology
Molenaar, P. C. M., & Campbell, C. G. (2009) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2009.01619.x · Cited by 1,088
IV.—On Least Squares and Linear Combination of Observations
Aitken, A. C. (1936) · Proceedings of the Royal Society of Edinburgh · doi:10.1017/s0370164600014346 · Cited by 950
Effect Size in Single-Case Research: A Review of Nine Nonoverlap Techniques
Parker, R. I., Vannest, K. J., & Davis, J. L. (2011) · Behavior Modification · doi:10.1177/0145445511399147 · Cited by 841
Maydeu‐Olivares, A., & McArdle, J. J. (2005) · Psychology Press eBooks · doi:10.4324/9781410612977 · Cited by 828
Single-case experimental designs: a systematic review of published research and current standards
Smith, J. D. (2012) · Psychological Methods · doi:10.1037/a0029312 · Cited by 630 · cited 3×
“This journal was selected because it is a leading journal on the topic used by applied researchers, and it strongly promotes the use of VA rather than quantitative analysis methods (Shadish & Sullivan, 2011; Smith, 2012).”
Design and Analysis of Time-Series Experiments.
Newbold, P., Glass, G. V., Willson, V. L., & Gottman, J. M. (1976) · Journal of the American Statistical Association · doi:10.2307/2285785 · Cited by 595
Kratochwill, T. R., & Levin, J. R. (2010) · Psychological Methods · doi:10.1037/a0017736 · Cited by 497
Huitema, B. E. (2011) · Wiley Series in Probability and Statistics · doi:10.1002/9781118067475 · Cited by 464
Characteristics of single-case designs used to assess intervention effects in 2008
Shadish, W. R., & Sullivan, K. J. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0111-y · Cited by 367
A standardized mean difference effect size for single case designs
Hedges, L. V., Pustejovsky, J. E., & Shadish, W. R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1052 · Cited by 308
The Analysis and Presentation of Graphic Data
Parsonson, B. S., & Baer, D. M. (1978) · Elsevier eBooks · doi:10.1016/b978-0-12-425850-1.50009-0 · Cited by 306
“PERHAPS IT WOULD BE BETTER NOT TO KNOW EVERYTHING”1
Baer, D. M. (1977) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1977.10-167 · Cited by 279
Peter, C. C. S., Vollmer, T. R., & Sloman, K. N. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-47 · Cited by 266
INCONSISTENT VISUAL ANALYSES OF INTRASUBJECT DATA
DeProspero, A., & Cohen, S. (1979) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1979.12-573 · Cited by 260
Interrupted time-series analysis with brief single-subject data.
Crosbie, J. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.61.6.966 · Cited by 248
Molenaar, P. C. M. (2008) · Developmental Psychobiology · doi:10.1002/dev.20262 · Cited by 237
Matyas, T. A., & Greenwood, K. M. (1990) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1990.23-341 · Cited by 222
Multilevel Meta-analysis: a comparison with traditional meta-analytical procedures
Van Den Noortgate, W., & Onghena, P. (2003) · Educational and Psychological Measurement · doi:10.1177/0013164403251027 · Cited by 218
Brossart, D. F., Parker, R. I., Olson, E. A., & Mahadevan, L. (2006) · Behavior Modification · doi:10.1177/0145445503261167 · Cited by 217
AN INVESTIGATION OF DIFFERENTIAL REINFORCEMENT OF ALTERNATIVE BEHAVIOR WITHOUT EXTINCTION
Athens, E. S., & Vollmer, T. R. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-569 · Cited by 212
Combining single-case experimental data using hierarchical linear models.
Noortgate, W. V. D., & Onghena, P. (2003) · School Psychology Quarterly · doi:10.1521/scpq.18.3.325.22577 · Cited by 195
A multilevel meta-analysis of single-subject experimental design studies
Van den Noortgate, W., & Onghena, P. (2008) · Evidence-Based Communication Assessment and Intervention · doi:10.1080/17489530802505362 · Cited by 192
Hierarchical linear models for the quantitative integration of effect sizes in single-case research
Noortgate, W. V. D., & Onghena, P. (2003) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195492 · Cited by 190
TIME‐SERIES ANALYSIS IN OPERANT RESEARCH1
Jones, R. R., Vaught, R. S., & Weinrott, M. R. (1977) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1977.10-151 · Cited by 187
EFFECTS OF SERIAL DEPENDENCY ON THE AGREEMENT BETWEEN VISUAL AND STATISTICAL INFERENCE1
Jones, R. R., Weinrott, M. R., & Vaught, R. S. (1978) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1978.11-277 · Cited by 186
Huitema, B. E. (2011) · Cited by 183
Cross-sectional time-series experiments: Some suggested statistical analyses.
Simonton, D. K. (1977) · Psychological Bulletin · doi:10.1037/0033-2909.84.3.489 · Cited by 164
First‐order integer valued AR processes with zero inflated poisson innovations
Jazi, M. A., Jones, G., & Lai, C. (2012) · Journal of Time Series Analysis · doi:10.1111/j.1467-9892.2012.00809.x · Cited by 158
Maggin, D. M., Swaminathan, H., Rogers, H. J., O’Keeffe, B. V., Sugai, G. M., & Horner, R. H. (2011) · Journal of School Psychology · doi:10.1016/j.jsp.2011.03.004 · Cited by 138
EFFECTS OF VIDEO MODELING ON TREATMENT INTEGRITY OF BEHAVIORAL INTERVENTIONS
Reed, F. D. D., Codding, R. S., Catania, C. N., & Maguire, H. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-291 · Cited by 114
Choe, G. H. (2005) · Springer eBooks · doi:10.1007/b138894 · Cited by 97
Carbone, V. J., Sweeney‐Kerwin, E. J., Attanasio, V., & Kasper, T. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-705 · Cited by 92
Telemetric monitoring in the behavior sciences
Goodwin, M. S., Velicer, W. F., & Intille, S. S. (2008) · Behavior Research Methods · doi:10.3758/brm.40.1.328 · Cited by 91
Self-citationComparing N = 1 Effect Size Indices in Presence of Autocorrelation
Manolov, R., & Solanas, A. (2008) · Behavior Modification · doi:10.1177/0145445508318866 · Cited by 88
Using UnGraph to extract data from image files: Verification of reliability and validity
Shadish, W. R., Brasil, I. C. C., Illingworth, D. A., White, K. D., Galindo, R. J., Nagler, E. D., & Rindskopf, D. M. (2009) · Behavior Research Methods · doi:10.3758/brm.41.1.177 · Cited by 87
The Aggregation of single-case results using hierarchical linear models
Van den Noortgate, W., & Onghena, P. (2007) · The Behavior Analyst Today · doi:10.1037/h0100613 · Cited by 86
BEHAVIIORAL COACHING TO IMPROVE OFFENSIVE LINE PASS‐BLOCKING SKILLS OF HIGH SCHOOL FOOTBALL ATHLETES
Stokes, J. V., Luiselli, J. K., Reed, D. D., & Fleming, R. K. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-463 · Cited by 85
Mevers, J. L., Fisher, W. W., & Kelley, M. E. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-425 · Cited by 78
Visual inspection of single-subject data: an empirical analysis.
Ottenbacher, K. J. (1990) · PubMed · Cited by 78
Irrelevant autocorrelation in least-squares intervention models.
Huitema, B. E., & McKean, J. W. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.1.104 · Cited by 76
Raiff, B. R., & Dallery, J. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-487 · Cited by 75
A Comparison of Alternative Approaches to the Analysis of Interrupted Time-Series
Harrop, J. W., & Velicer, W. F. (1985) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2001_2 · Cited by 73
Self-citationASSESSMENT AND TREATMENT OF ELOPEMENT MAINTAINED BY ACCESS TO STEREOTYPY
Falcomata, T. S., Roane, H. S., Feeney, B. J., & Stephenson, K. M. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-513 · Cited by 70
DISCRIMINATED FUNCTIONAL COMMUNICATION: A PROCEDURAL EXTENSION OF FUNCTIONAL COMMUNICATION TRAINING
Kuhn, D. E., Chirighin, A. E., & Zelenka, K. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-249 · Cited by 69
The Reliability and Accuracy of Time Series Model Identification
Velicer, W. F., & Harrop, J. W. (1983) · Evaluation Review · doi:10.1177/0193841x8300700408 · Cited by 66
Self-citationToussaint, K. A., & Tiger, J. H. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-181 · Cited by 65
USING COMPLEX AUDITORY—VISUAL SAMPLES TO PRODUCE EMERGENT RELATIONS IN CHILDREN WITH AUTISM
Groskreutz, N. C., Karsina, A., Miguel, C. F., & Groskreutz, M. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-131 · Cited by 64
When the Truth Hits You Between the Eyes
Bulté, I., & Onghena, P. (2011) · Methodology · doi:10.1027/1614-2241/a000042 · Cited by 55
Ülke-Kürkçüoğlu, B., & Kırcaali-İftar, G. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-717 · Cited by 52
ON THE RELATION BETWEEN REINFORCER EFFICACY AND PREFERENCE
Lee, M. S. H., Yu, C. T., Martin, T. L., & Martin, G. L. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-95 · Cited by 52
A SIMPLE AND POWERFUL TEST FOR AUTOCORRELATED ERRORS IN OLS INTERVENTION MODELS
Huitema, B. E., & McKean, J. W. (2000) · Psychological Reports · doi:10.2466/pr0.2000.87.1.3 · Cited by 48
FUNCTIONAL ANALYSIS AND TREATMENT OF AGGRESSION MAINTAINED BY PREFERRED CONVERSATIONAL TOPICS
Roscoe, E. M., Kindle, A. E., & Pence, S. T. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-723 · Cited by 44
Times Series Analysis Without Model Identification
Velicer, W. F., & McDonald, R. P. (1984) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr1901_2 · Cited by 42
Self-citationCarter, S. L. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-543 · Cited by 38
ANALYSIS OF DATA IN IDIOGRAPHIC RESEARCH
Ottenbacher, K. J. (1992) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/00002060-199208000-00002 · Cited by 38
Smith, J. D., Borckardt, J. J., & Nash, M. R. (2011) · Behavior Therapy · doi:10.1016/j.beth.2011.10.001 · Cited by 35
A BEHAVIORAL INTERVENTION FOR TEACHING TACKLING SKILLS TO HIGH SCHOOL FOOTBALL ATHLETES
Stokes, J. V., Luiselli, J. K., & Reed, D. D. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-509 · Cited by 33
Dolezal, D. N., & Kurtz, P. F. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-309 · Cited by 33
USE OF PEER‐MEDIATED INTERVENTION IN CHILDREN WITH ATTENTION DEFICIT HYPERACTIVITY DISORDER
MacAleese, A. N., & Wallace, M. D. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-547 · Cited by 31
Molenaar, P. C. M. (2008) · Sage eBooks · doi:10.4135/9781412976589.n5 · Cited by 29
Wilder, D. A., Allison, J., Nicholson, K., Abellon, O. E., & Saulnier, R. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-601 · Cited by 28
Wu, J. (2012) · Chapman & Hall/CRC eBooks · doi:10.1201/b11639-10 · Cited by 27
AN EVALUATION OF ADVANCE NOTICE TO INCREASE COMPLIANCE AMONG PRESCHOOLERS
Wilder, D. A., Nicholson, K., & Allison, J. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-751 · Cited by 26
FURTHER EXAMINATION OF DISCRIMINATED FUNCTIONAL COMMUNICATION
León, Y., Hausman, N. L., Kahng, S., & Becraft, J. L. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-525 · Cited by 25
The Effect of Autocorrelation on the Results of Visually Analyzing Data From Single-Subject Designs
Bengali, M. K., & Ottenbacher, K. J. (1998) · American Journal of Occupational Therapy · doi:10.5014/ajot.52.8.650 · Cited by 24
INCREASING SEAT BELT USE IN SERVICE VEHICLE DRIVERS WITH A GEARSHIFT DELAY
Houten, R. V., Malenfant, J. E. L., Reagan, I. J., Sifrit, K. J., Compton, R., & Tenenbaum, J. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-369 · Cited by 22
Travis, R. W., & Sturmey, P. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-745 · Cited by 21
Time-Series Intervention Analysis Using ITSACORR: Fatal Flaws
Huitema, B. E., McKean, J. W., & Laraway, S. (2007) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1193889780 · Cited by 18
AN EXPERIMENTAL ANALYSIS OF NEGATIVE REINFORCEMENT CONTINGENCIES FOR ADULT‐DELIVERED REPRIMANDS
Miller, J. R., Lerman, D. C., & Fritz, J. N. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-769 · Cited by 18
THE EFFECTS OF FIXED‐TIME ESCAPE ON INAPPROPRIATE AND APPROPRIATE CLASSROOM BEHAVIOR
Waller, R. D., & Higbee, T. S. (2010) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2010.43-149 · Cited by 17
Computer Programs for Interrupted Time Series Analysis: II A Quantitative Evaluation
Harrop, J. W., & Velicer, W. F. (1990) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2502_13 · Cited by 15
Self-citationComputer Programs for Interrupted Time Series Analysis: I. A Qualitative Evaluation
Harrop, J. W., & Velicer, W. F. (1990) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2502_12 · Cited by 6
Self-citation
Research works printed but not linked: 5
- Bengali M.K., 1998, Quantitative Research Series, 52, 650
- Biosoft, 2004, UnGraph®for Windows
- Ottenbacher K.J., 1990, Mental Retardation, 28, 283
- Velicer W.F., 2014, Handbook of Relational Developmental Systems Theory and Methodology, 425
- Velicer W.F., 2013, Handbook of Psychology: Research Methods in Psychology, 50, 628
Cited by
Sorted by citations
Ewusie, J. E., Soobiah, C., Blondal, E., Beyene, J., Thabane, L., & Hamid, J. S. (2020) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s241085 · Cited by 170
Recommendations for Choosing Single-Case Data Analytical Techniques.
Manolov, R., & Moeyaert, M. (2017) · The Behavior Therapist · doi:10.1016/j.beth.2016.04.008 · Cited by 122
How Can Single-Case Data Be Analyzed? Software Resources, Tutorial, and Reflections on Analysis
Manolov, R., & Moeyaert, M. (2016) · Behavior Modification · doi:10.1177/0145445516664307 · Cited by 91
Randomized single-case AB phase designs: Prospects and pitfalls
Michiels, B., & Onghena, P. (2019) · Behavior Research Methods · doi:10.3758/s13428-018-1084-x · Cited by 77
Violin plots as visual tools in the meta-analysis of Single-Case Experimental Designs
Tanious, R., & Manolov, R. (2022) · Methodology · doi:10.5964/meth.9209 · Cited by 58
Metadata
- Authors
- Harrington, M., & Velicer, W. F.
- Affiliations
- University of Rhode Island (US)
- Corresponding author
- Velicer, Wayne F.
- ORCID
- Velicer, Wayne F.
- Venue
- Multivariate Behavioral Research
- Publisher
- Informa UK Limited
- Published
- 2015-03-04
- Volume
- 50
- Issue
- 2
- Pages
- 162-183
- ISSN
- 0027-3171 (ISSN-L)
- 1532-7906
- PMC
- PMC4677800
- OpenAlex
- W1969454087
- Cited by
- 114
- References
- 80
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 14 · 2023: 9 · 2022: 18 · 2021: 18 · 2020: 16 · 2019: 6 · 2018: 7 · 2017: 10 · 2016: 6
- FWCI
- 19.39
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Multivariate Behavioral Research (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Journal of Media Literacy Education (submittedVersion, cc-by)
- Funders
- National Institute on Drug Abuse (R01DA020112, DA020112)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- interrupted time series, visual analysis, single-case experimental design, autocorrelation, effect size, applied behavior analysis, measurement agreement
- MeSH terms
- Humans, Research Design, Reproducibility of Results, Behavioral Research, Interrupted Time Series Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review