CoChat
Sign Up
Cited by 88Psychometric Methodologies and TestingEvaluation and Performance AssessmentStatistical Methods and Bayesian Inference

Evaluating Intervention Programs with a Pretest-Posttest Design: A Structural Equation Modeling Approach

Open access
Alessandri, Guido, Zuffianò, Antonio, Perinelli, Enrico·2017·Frontiers in Psychology
DOI 10.3389/fpsyg.2017.00223PMC PMC5332425
Open this paper in CoChat — free accountOpen PDF

Abstract

A common situation in the evaluation of intervention programs is the researcher's possibility to rely on two waves of data only (i.e., pretest and posttest), which profoundly impacts on his/her choice about the possible statistical analyses to be conducted. Indeed, the evaluation of intervention programs based on a pretest-posttest design has been usually carried out by using classic statistical tests, such as family-wise ANOVA analyses, which are strongly limited by exclusively analyzing the intervention effects at the group level. In this article, we showed how second order multiple group latent curve modeling (SO-MG-LCM) could represent a useful methodological tool to have a more realistic and informative assessment of intervention programs with two waves of data. We offered a practical step-by-step guide to properly implement this methodology, and we outlined the advantages of the LCM approach over classic ANOVA analyses. Furthermore, we also provided a real-data example by re-analyzing the implementation of the Young Prosocial Animation, a universal intervention program aimed at promoting prosociality among youth. In conclusion, albeit there are previous studies that pointed to the usefulness of MG-LCM to evaluate intervention programs (Muthén and Curran, 1997; Curran and Muthén, 1999), no previous study showed that it is possible to use this approach even in pretest-posttest (i.e., with only two time points) designs. Given the advantages of latent variable analyses in examining differences in interindividual and intraindividual changes (McArdle, 2009), the methodological and substantive implications of our proposed approach are discussed.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fpsyg.2017.00223

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The analysis of pre- and posttest data can be subject to issues with regression to the mean, such as when survey items register initially high responses, which may limit the effectiveness of tests of significant difference.

    “the analysis of pre- and posttest data can be subject to issues with regression to the mean, such as when survey items register initially high responses, which may limit the effectiveness of tests of significant difference (Alessandri et al., 2017)”

    From AI Literacy in the Communication Basic Course: Insights on Embedding, Motivation, and Reducing Uncertainty (2026) · cites this paper as Alessandri et al., 2017

    The Discussion reports that participants with high pretest prosociality remained at a high level, while those with lower initial levels showed greater gains. The paper also discusses limitations of conventional significance tests for pretest–posttest data, but it does not identify regression to the mean, or initially high survey responses causing it, as a reason those tests may be less effective. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 38

  1. Achenbach T. M. (2017). Future directions for clinical research, services, and training: evidence-based assessment across informants, cultures, and dimensional hierarchies. J. Clin. Child Adolesc. Psychol. 46, 159– 169. 10.1080/15374416.2016.1220315 27736239 · DOI
  2. Alessandri G. Vecchione M. Caprara G. V. Letzring T. D. (2012). The ego resiliency scale revised: a crosscultural study in Italy, Spain, and the United States. Eur. J. Psychol. Assess. 28, 139– 146. 10.1027/1015-5759/a000102 · DOI
  3. Bandura A. (1977). Self-efficacy: toward a unifying theory of behavioral change. Psychol. Rev. 84, 191– 215. 10.1037/0033-295X.84.2.191 847061 · DOI · Paper page
  4. Bishop J. Geiser C. Cole D. A. (2015). Modeling latent growth with multiple indicators: a comparison of three approaches. Psychol. Methods 20, 43– 62. 10.1037/met0000018 24885340 · DOI
  5. Bollen K. A. (1989). Structural Equations with Latent Variables. New York, NY: Wiley.
  6. Bollen K. A. Curran P. J. (2006). Latent Curve Models: A Structural Equation Perspective. Hoboken, NJ: Wiley.
  7. Brown T. A. (2015). Confirmatory Factor Analysis for Applied Research. New York, NY: The Guilford Press.
  8. Burnham K. P. Anderson D. R. (2004). Multimodel inference: understanding AIC and BIC in model selection. Sociol. Methods Res. 33, 261– 304. 10.1177/0049124104268644 · DOI · Paper page
  9. Caprara G. V. Alessandri G. Eisenberg N. (2012). Prosociality: the contribution of traits, values, and self-efficacy beliefs. J. Pers. Soc. Psychol. 102, 1289– 1303. 10.1037/a0025626 21942280 · DOI
  10. Caprara G. V. Luengo Kanacri B. P. Gerbino M. Zuffianò A. Alessandri G. Vecchio G. (2014). Positive effects of promoting prosocial behavior in early adolescents: evidence from a school-based intervention. Int. J. Behav. Dev. 4, 386– 396. 10.1177/0165025414531464 · DOI
  11. Caprara G. V. Steca P. Zelli A. Capanna C. (2005). A new scale for measuring adults' prosocialness. Eur. J. Psychol. Assess. 21, 77– 89. 10.1027/1015-5759.21.2.77 · DOI
  12. Cole D. A. Preacher K. J. (2014). Manifest variable path analysis: potentially serious and misleading consequences due to uncorrected measurement error. Psychol. Methods 19, 300– 315. 10.1037/a0033805 24079927 · DOI
  13. Curran P. J. Muthén B. O. (1999). The application of latent curve analysis to testing developmental theories in intervention research. Am. J. Commun. Psychol. 27, 567– 595. 10.1023/A:1022137429115 10573835 · DOI
  14. Curran P. J. West S. G. Finch J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychol. Methods 1, 16– 29. 10.1037/1082-989X.1.1.16 · DOI · Paper page
  15. Eisenberg N. Spinrad T. L. Knafo-Noam A. (2015). Prosocial development, in Handbook of Child Psychology and Developmental Science Vol. 3, 7th Edn., eds Lamb M. E. Lerner R. M. (Hoboken, NJ: Wiley), 610– 656.
  16. Enders C. K. (2010). Applied Missing Data Analysis. New York, NY: Guilford Press.
  17. Geiser C. Keller B. T. Lockhart G. (2013). First-versus second-order latent growth curve models: some insights from latent state-trait theory. Struct. Equ. Modeling 20, 479– 503. 10.1080/10705511.2013.797832 24244087 PMC3827780 · DOI
  18. Greenberg M. T. Domitrovich C. Bumbarger B. (2001). The prevention of mental disorders in school-aged children: current state of the field. Prevent. Treat. 4: 1a 10.1037/1522-3736.4.1.41a · DOI
  19. Kline R. B. (2016). Principles and Practice of Structural Equation Modeling, 4th Edn. New York, NY: The Guilford Press.
  20. Little T. D. (1997). Mean and covariance structures (MACS) analyses of cross-cultural data: practical and theoretical issues. Multivariate Behav. Res. 32, 53– 76. 10.1207/s15327906mbr3201_3 26751106 · DOI
  21. Little T. D. (2013). Longitudinal Structural Equation Modeling. New York, NY: The Guilford Press.
  22. Little T. D. Card N. A. Preacher K. J. McConnell E. (2009). Modeling longitudinal data from research on adolescence, in Handbook of Adolescent Psychology, Vol. 2, 3rd Edn., eds Lerner R. M. Steinberg L. (Hoboken, NJ: Wiley), 15– 54.
  23. Little T. D. Cunningham W. A. Shahar G. Widaman K. F. (2002). To parcel or not to parcel: exploring the question, weighing the merits. Struct. Equ. Modeling 9, 151– 173. 10.1207/S15328007SEM0902_1 · DOI · Paper page
  24. Lucas R. E. Donnellan M. B. (2012). Estimating the reliability of single-item life satisfaction measures: results from four national panel studies. Soc. Indic. Res. 105, 323– 331. 10.1007/s11205-011-9783-z 23087538 PMC3475500 · DOI · Paper page
  25. Malti T. Noam G. G. Beelmann A. Sommer S. (2016). Good Enough? Interventions for child mental health: from adoption to adaptation—from programs to systems. J. Clin. Child Adolesc. Psychol. 45, 707– 709. 10.1080/15374416.2016.1157759 27310680 · DOI
  26. Mecklin C. J. Mundfrom D. J. (2005). A Monte Carlo comparison of the Type I and Type II error rates of tests of multivariate normality. J. Stat. Comput. Simul. 75, 93– 107. 10.1080/0094965042000193233 · DOI
  27. Meredith W. Teresi J. A. (2006). An essay on measurement and factorial invariance. Med. Care 44, S69– S77. 10.1097/01.mlr.0000245438.73837.89 17060838 · DOI · Paper page
  28. Meredith W. Tisak J. (1990). Latent curve analysis. Psychometrika 55, 107– 122. 10.1007/BF02294746 · DOI
  29. Micceri T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychol. Bull. 105, 156– 166. 10.1037/0033-2909.105.1.156 · DOI
  30. Millsap R. E. (2011). Statistical Approaches to Measurement Invariance. New York, NY: Routledge.
  31. Muthén B. O. Curran P. J. (1997). General longitudinal modeling of individual differences in experimental designs: a latent variable framework for analysis and power estimation. Psychol. Methods 2, 371– 402. 10.1037/1082-989X.2.4.371 · DOI
  32. Nimon K. F. (2012). Statistical assumptions of substantive analyses across the general linear model: a mini-review. Front. Psychol. 3: 322. 10.3389/fpsyg.2012.00322 22973253 PMC3428812 · DOI
  33. Roberts M. C. Ilardi S. S. (2003). Handbook of Research Methods in Clinical Psychology. Oxford: Blackwell Publishing.
  34. Schmider E. Ziegler M. Danay E. Beyer L. Bühner M. (2010). Is it really robust? Reinvestigating the robustness of ANOVA against violations of the normal distribution assumption. Methodology 6, 147– 151. 10.1027/1614-2241/a000016 · DOI
  35. Tabachnick B. G. Fidell L. S. (2013). Using Multivariate Statistics, 6th Edn. New Jersey, NJ: Pearson.
  36. Villasenor Alva J. A. Estrada E. G. (2009). A generalization of Shapiro–Wilk's test for multivariate normality. Commun. Stat. Theor. Methods 38, 1870– 1883. 10.1080/03610920802474465 · DOI
  37. Wilcox R. R. (1998). The goals and strategies of robust methods. Br. J. Math. Stat. Psychol. 51, 1– 39. 10.1111/j.2044-8317.1998.tb00659.x · DOI
  38. Wolf E. J. Harrington K. M. Clark S. L. Miller M. W. (2013). Sample size requirements for structural equation models: an evaluation of power, bias, and solution propriety. Educ. Psychol. Meas. 76, 913– 934. 10.1177/0013164413495237 25705052 PMC4334479 · DOI · Paper page

Cited by

Sorted by citations

Authors
Alessandri, G., Zuffianò, A., & Perinelli, E.
Affiliations
  • Sapienza University of Rome (IT)
  • Liverpool Hope University (GB)
Corresponding author
Alessandri, Guido
ORCID
Venue
Frontiers in Psychology
Publisher
Frontiers Media SA
Published
2017-03-02
Volume
8
Pages
223-223
ISSN
1664-1078 (ISSN-L)
DOI
10.3389/fpsyg.2017.00223
PMC
PMC5332425
OpenAlex
W2594804352
Cited by
88
References
2
Citations by year
2026: 14 · 2025: 20 · 2024: 14 · 2023: 11 · 2022: 7 · 2021: 6 · 2020: 10 · 2019: 5 · 2018: 1
FWCI
3.63
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Frontiers in Psychology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Hope's Institutional Research Archive (Liverpool Hope University) (acceptedVersion)
  • IRIS Research product catalog (Sapienza University of Rome) (submittedVersion, cc-by)
  • Institutional Research Information System (Università degli Studi di Trento) (submittedVersion, cc-by)
Funders
Sapienza Università di Roma
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords
intervention evaluation, pretest-posttest design, intervention effects, individual differences, within-person change, prosocial behavior, youth

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review