Moving From the Lab to the Field: The Role of Fidelity and Achieved Relative Intervention Strength
Abstract
As an example of how the effectiveness of well-designed laboratory interventions is often diffused in the field, we examined the effects of a motivation intervention on students' perceptions and learning. The intervention proved to be more effective in the laboratory (g = 0.45) than the field (g = 0.05) in enhancing subsequent motivation. We explored this reduction in treatment effectiveness through a fidelity analysis that examined the extent to which participants responded to the treatment. We calculated fidelity as three indices of achieved relative treatment strength (Cordray & Pion, 2006 Cordray, D. S. . The cause … or the ‘what’ of what works?. Paper presented at the Institute for Education Sciences 2006 Research Conference. Washington, DC. June, [Google Scholar]), and found that, regardless of how fidelity was calculated, achieved relative strength was about 1 standard deviation less in the classroom than the laboratory. In addition, greater levels of achieved relative strength were associated with greater differences in the motivational outcome—indicating that intervention was more effective for participants who actually received the treatment than those who did not. Multilevel analyses indicated that the drop in classroom treatment fidelity was partially because of teacher, rather than student, factors. Implications for theoretical models of change and research are discussed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Field interventions are generally more difficult to implement than interventions under laboratory conditions.
“field interventions are generally more difficult to implement than interventions under laboratory conditions (Hulleman and Cordray 2009).”
From Transfer of metacognitive skills in self-regulated learning: an experimental training study (2020) · cites this paper as Hulleman and Cordray 2009
Cites
Hierarchical Linear Models: Applications and Data Analysis Methods.
Longford, N. T., Bryk, A. S., & Raudenbush, S. W. (1993) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2075823 · Cited by 18,768 · see citing works
Hierarchical Linear Models: Applications and Data Analysis Methods
Davidian, M. (2003) · Journal of the American Statistical Association · doi:10.1198/jasa.2003.s288 · Cited by 13,820 · see citing works
(2002) · Cited by 13,211 · see citing works
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293 · see citing works
Identification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230 · see citing works
Expectancies, values and academic behaviors
Eccles, J. S. (1983) · Publication Server of Goethe University Frankfurt am Main (Goethe University Frankfurt) · Cited by 3,901 · see citing works
(2005) · Cited by 3,748 · see citing works
Dusenbury, L. (2003) · Health Education Research · doi:10.1093/her/18.2.237 · Cited by 1,671 · see citing works
Dane, A. V., & Schneider, B. H. (1998) · Clinical Psychology Review · doi:10.1016/s0272-7358(97)00043-3 · Cited by 1,557 · cited 2× · see citing works
“The notion of intervention fidelity has been captured under a broad array of labels, such as treatment integrity, adherence, compliance, dose, exposure, quality of delivery, and treatment differentiation (for reviews, see Dane & Schneider, 1998 ; Dusenbury, Brannigan, Falco, & Hansen, 2003 ; Mowbray, Holter, Teague, &…”
BackgroundO’Donnell, C. (2008) · Review of Educational Research · doi:10.3102/0034654307313793 · Cited by 1,037 · see citing works
Fidelity Criteria: Development, Measurement, and Validation
Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. I. (2003) · American Journal of Evaluation · doi:10.1177/109821400302400303 · Cited by 884 · see citing works
Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence
Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.61.4.620 · Cited by 873 · see citing works
Effect Sizes in Cluster-Randomized Designs
Hedges, L. (2007) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998606298043 · Cited by 605 · see citing works
Accounting for No-Shows in Experimental Evaluation Designs
Bloom, H. S. (1984) · Evaluation Review · doi:10.1177/0193841x8400800205 · Cited by 532 · cited 2× · see citing works
“…approach is often utilized to estimate the effects of an intervention for only those treatment participants who actually received the intervention, that is, complier average causal effect estimation (e.g., Angrist, Imbens, & Rubin, 1996; Bloom, 1984; Jo, Asparouhov, Muthen, Ialongo, & Brown, 2008).”
BackgroundYeaton, W. H., & SECHREST, L. B. (1981) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.49.2.156 · Cited by 501 · see citing works
Task values, achievement goals, and interest: An integrative analysis.
Hulleman, C. S., Durik, A. M., Schweigert, S. A., & Harackiewicz, J. M. (2008) · Journal of Educational Psychology · doi:10.1037/0022-0663.100.2.398 · Cited by 498 · see citing works
Self-citationYeaton, W. H., & SECHREST, L. B. (1981) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.49.2.156 · Cited by 407 · see citing works
Achievement and Achievement Motives: Psychological and Sociological Approaches
Ferguson, E. D., & SPENCE, J. T. (1984) · The American Journal of Psychology · doi:10.2307/1422172 · Cited by 383 · see citing works
Malka, A., & Covington, M. V. (2004) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2004.04.001 · Cited by 126 · see citing works
Sensitivity, bias, and theory in impact evaluations.
Boruch, R. F., & Gomez, H. (1977) · Professional Psychology · doi:10.1037/0735-7028.8.4.411 · Cited by 108 · see citing works
Gottfredson, G. D. (1984) · American Psychologist · doi:10.1037/0003-066x.39.10.1101 · Cited by 108 · see citing works
Simons, J., Dewitte, S., & Lens, W. (2003) · Journal of Sport and Exercise Psychology · doi:10.1123/jsep.25.2.145 · Cited by 92 · see citing works
Cluster randomized trials with treatment noncompliance.
Jo, B., Asparouhov, T., Muthén, B. O., Ialongo, N. S., & Brown, C. H. (2008) · Psychological Methods · doi:10.1037/1082-989x.13.1.1 · Cited by 83 · see citing works
Treatment Strength and Integrity: Models and Methods.
Cordray, D. S., & Pion, G. M. (2006) · American Psychological Association eBooks · doi:10.1037/11384-006 · Cited by 74 · see citing works
Self-citationThe Role of Utility Value in the Development of Interest and Achievement.
Hulleman, C. S. (2007) · Cited by 18 · see citing works
Self-citationConstructing Quantitative Implementation Scales From Categorical Services Data
Orwin, R. G., Sonnefeld, L. J., Cordray, D. S., Pion, G. M., & Perl, H. I. (1998) · Evaluation Review · doi:10.1177/0193841x9802200204 · Cited by 12 · see citing works
Self-citationThe scaling of reading interventions: Building multilevel insight
Sloane, F. C. (2005) · Reading Research Quarterly · doi:10.1598/rrq.40.3.4 · Cited by 9 · see citing works
Flagged first, then by citations
2 of the 29 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Social-Psychological Interventions in Education: They’re Not Magic
Yeager, D. S., & Walton, G. M. (2011) · Review of Educational Research · doi:10.3102/0034654311405999 · Cited by 1,345 · see citing works
Why Interventions to Influence Adolescent Behavior Often Fail but Could Succeed
Yeager, D. S., Dahl, R. E., & Dweck, C. S. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691617722620 · Cited by 512 · see citing works
Motivation Interventions in Education: A Meta-Analytic Review
Lazowski, R. A., & Hulleman, C. S. (2016) · Review of Educational Research · doi:10.3102/0034654315617832 · Cited by 607 · see citing works
Self-citationImproving Student Outcomes in Higher Education: The Science of Targeted Intervention
Harackiewicz, J. M., & Priniski, S. J. (2018) · Annual Review of Psychology · doi:10.1146/annurev-psych-122216-011725 · Cited by 429 · see citing works
Making connections: Replicating and extending the utility value intervention in the classroom.
Hulleman, C. S., Kosovich, J. J., Barron, K. E., & Daniel, D. B. (2016) · Journal of Educational Psychology · doi:10.1037/edu0000146 · Cited by 370 · see citing works
Self-citationFostering adolescents' value beliefs for mathematics with a relevance intervention in the classroom.
Gaspard, H., Dicke, A.-L., Flunger, B., Brisson, B., Häfner, I., Nagengast, B., & Trautwein, U. (2015) · Developmental Psychology · doi:10.1037/dev0000028 · Cited by 352 · see citing works
Clements, D. H., Sarama, J. H., Spitler, M. E., Lange, A. A., & Wolfe, C. B. (2011) · Journal for Research in Mathematics Education · doi:10.5951/jresematheduc.42.2.0127 · Cited by 325 · see citing works
Strengthening school readiness for Head Start children: Evaluation of a self-regulation intervention
Schmitt, S. A., McClelland, M. M., Tominey, S. L., & Acock, A. C. (2014) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2014.08.001 · Cited by 304 · see citing works
Century, J., & Cassata, A. (2016) · Review of Research in Education · doi:10.3102/0091732x16665332 · Cited by 248 · see citing works
Knogler, M., Harackiewicz, J. M., Gegenfurtner, A., & Lewalter, D. (2015) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2015.08.004 · Cited by 219 · see citing works
Efficacy of the Responsive Classroom Approach
Rimm‐Kaufman, S. E., Larsen, R., Baroody, A. E., Curby, T. W., Ko, M., Thomas, J., Merritt, E. G., Abry, T., & DeCoster, J. (2014) · American Educational Research Journal · doi:10.3102/0002831214523821 · Cited by 196 · see citing works
Nelson, M. C., Cordray, D. S., Hulleman, C. S., Darrow, C. L., & Sommer, E. C. (2012) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-012-9295-x · Cited by 192 · see citing works
Self-citationA CONCEPTUAL FRAMEWORK FOR STUDYING THE SOURCES OF VARIATION IN PROGRAM EFFECTS
Weiss, M. J., Bloom, H. S., & Brock, T. (2013) · Journal of Policy Analysis and Management · doi:10.1002/pam.21760 · Cited by 175 · see citing works
Interleaved practice improves mathematics learning.
Rohrer, D., Dedrick, R. F., & Stershic, S. (2015) · Journal of Educational Psychology · doi:10.1037/edu0000001 · Cited by 164 · see citing works
Rosenzweig, E. Q., Wigfield, A., & Eccles, J. S. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1984242 · Cited by 176 · see citing works
Maggin, D. M., Zurheide, J., Pickett, K. C., & Baillie, S. J. (2015) · Journal of Positive Behavior Interventions · doi:10.1177/1098300715573630 · Cited by 141 · see citing works
Clinical trial of Second Step© middle-school program: Impact on aggression & victimization
Espelage, D. L., Low, S. K., Polanin, J. R., & Brown, E. C. (2015) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2014.11.007 · Cited by 141 · see citing works
Fidelity: An Essential Component of Evidence-Based Practice in Speech-Language Pathology
Kaderavek, J. N., & Justice, L. M. (2010) · American Journal of Speech-Language Pathology · doi:10.1044/1058-0360(2010/09-0097 · Cited by 137 · see citing works
Dozier, M., & Bernard, K. (2017) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2017.03.003 · Cited by 133 · see citing works
Harnessing Values to Promote Motivation in Education
Harackiewicz, J. M., Tibbetts, Y., Canning, E., & Hyde, J. S. (2014) · Motivational Interventions · doi:10.1108/s0749-742320140000018002 · Cited by 131 · see citing works
Snyder, F., Flay, B., Vuchinich, S., Acock, A., Washburn, I., Beets, M., & Li, K.-K. (2009) · Journal of Research on Educational Effectiveness · doi:10.1080/19345740903353436 · Cited by 130 · see citing works
Gaspard, H., Dicke, A., Flunger, B., Häfner, I., Brisson, B. M., Trautwein, U., & Nagengast, B. (2016) · AERA Open · doi:10.1177/2332858416649168 · Cited by 126 · see citing works
Caron, E., Bernard, K., & Dozier, M. (2016) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2016.1141359 · Cited by 124 · see citing works
What Really Works in Intervention? Using Fidelity Measures to Support Optimal Outcomes
An, M., Dusing, S. C., Harbourne, R. T., & Sheridan, S. M. (2020) · Physical Therapy · doi:10.1093/ptj/pzaa006 · Cited by 123 · see citing works
Light, lighting and human health
Boyce, P. R. (2021) · Lighting Research & Technology · doi:10.1177/14771535211010267 · Cited by 118 · see citing works
Naturalistic Instructional Approaches in Early Learning
Snyder, P., Rakap, S., Hemmeter, M. L., McLaughlin, T., Sandall, S. R., & McLean, M. (2015) · Journal of Early Intervention · doi:10.1177/1053815115595461 · Cited by 114 · see citing works
Effectiveness of Reading-Strategy Interventions in Whole Classrooms: a Meta-Analysis
Okkinga, M., van Steensel, R., van Gelderen, A. J. S., van Schooten, E., Sleegers, P. J. C., & Arends, L. R. (2018) · Educational Psychology Review · doi:10.1007/s10648-018-9445-7 · Cited by 108 · see citing works
Using Indices of Fidelity to Intervention Core Components to Identify Program Active Ingredients
Abry, T., Hulleman, C. S., & Rimm‐Kaufman, S. E. (2014) · American Journal of Evaluation · doi:10.1177/1098214014557009 · Cited by 106 · see citing works
Self-citationTransfer of metacognitive skills in self-regulated learning: an experimental training study
Schuster, C., Stebner, F., Leutner, D., & Wirth, J. (2020) · Metacognition and Learning · doi:10.1007/s11409-020-09237-5 · Cited by 105 · see citing works
Examining the Association Between Implementation and Outcomes
Pas, E. T., & Bradshaw, C. P. (2012) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-012-9290-2 · Cited by 103 · see citing works
Wehby, J. H., Maggin, D. M., Partin, T. C. M., & Robertson, R. E. (2011) · School Mental Health · doi:10.1007/s12310-011-9067-4 · Cited by 99 · see citing works
Brisson, B. M., Dicke, A.-L., Gaspard, H., Häfner, I., Flunger, B., Nagengast, B., & Trautwein, U. (2017) · American Educational Research Journal · doi:10.3102/0002831217716084 · Cited by 98 · see citing works
A randomized controlled trial of interleaved mathematics practice
Rohrer, D., Dedrick, R. F., Hartwig, M. K., & Cheung, C.-N. (2020) · Journal of Educational Psychology · doi:10.1037/edu0000367 · Cited by 90 · see citing works
Jeffries, P. R., Dreifuerst, K. T., Kardong‐Edgren, S., & Hayden, J. K. (2015) · Journal of Nursing Regulation · doi:10.1016/s2155-8256(15)30037-5 · Cited by 91 · see citing works
Baker, C. N., Kupersmidt, J. B., Voegler‐Lee, M. E., Arnold, D. H., & Willoughby, M. T. (2009) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2009.09.005 · Cited by 91 · see citing works
Hecht, C. A., Harackiewicz, J. M., Priniski, S. J., Canning, E. A., Tibbetts, Y., & Hyde, J. S. (2019) · Journal of Educational Psychology · doi:10.1037/edu0000356 · Cited by 90 · see citing works
Yoshikawa, H., Wuermli, A. J., Raikes, A., Kim, S., & Kabay, S. (2018) · Child Policy Nexus · doi:10.1002/j.2379-3988.2018.tb00091.x · Cited by 90 · see citing works
Abry, T., Rimm‐Kaufman, S. E., Larsen, R., & Brewer, A. J. (2013) · Journal of School Psychology · doi:10.1016/j.jsp.2013.03.001 · Cited by 85 · see citing works
Festas, I., de Oliveira, A. L., Rebelo, J. A., Damião, M. H., Harris, K. R., & Graham, S. (2014) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2014.05.004 · Cited by 83 · see citing works
Hecht, C. A., Yeager, D. S., Dweck, C. S., & Murphy, M. C. (2021) · Advances in child development and behavior · doi:10.1016/bs.acdb.2021.04.004 · Cited by 93 · see citing works
O’Connor, E. E., Cappella, E., McCormick, M. P., & McClowry, S. (2014) · Journal of Educational Psychology · doi:10.1037/a0036615 · Cited by 78 · see citing works
Weiland, C., McCormick, M. P., Mattera, S. K., Maier, M. F., & Morris, P. (2018) · AERA Open · doi:10.1177/2332858418757735 · Cited by 78 · see citing works
Flay, B. R., & Allred, C. G. (2010) · Springer eBooks · doi:10.1007/978-90-481-8675-4_28 · Cited by 73 · see citing works
Quinn, D. M., & Kim, J. S. (2017) · American Educational Research Journal · doi:10.3102/0002831217717692 · Cited by 74 · see citing works
Attachment and Biobehavioral Catch-Up
Costello, A. H., Roben, C. K. P., & Dozier, M. (2018) · Springer eBooks · doi:10.1007/978-3-030-03110-7_9 · Cited by 70 · see citing works
Improving Middle-School Students’ Knowledge and Comprehension in Social Studies: a Replication
Vaughn, S. R., Roberts, G., Swanson, E. A., Wanzek, J., Fall, A., & Stillman‐Spisak, S. J. (2014) · Educational Psychology Review · doi:10.1007/s10648-014-9274-2 · Cited by 67 · see citing works
Bleses, D., Højen, A., Justice, L. M., Dale, P. S., Dybdal, L., Piasta, S. B., Markussen‐Brown, J., Clausen, M. C., & Haghish, E. F. (2017) · Child Development · doi:10.1111/cdev.12859 · Cited by 68 · see citing works
Fogarty, M., Oslund, E. L., Simmons, D. C., Davis, J. L., Simmons, L. E., Anderson, L. L., Clemens, N. H., & Roberts, G. J. (2014) · Educational Psychology Review · doi:10.1007/s10648-014-9270-6 · Cited by 64 · see citing works
The Effects of Group differentiation by students’ learning strategies
Haelermans, C. (2022) · Instructional Science · doi:10.1007/s11251-021-09575-0 · Cited by 100 · see citing works
Fyfe, E. R., de Leeuw, J. R., Carvalho, P. F., Goldstone, R. L., Sherman, J., Admiraal, D., Alford, L. K., Bonner, A., Brassil, C. E., Brooks, C. A., et al. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211027575 · Cited by 59 · see citing works
Special Educator Burnout and Fidelity in Implementing Behavior Support Plans: A Call to Action
Garwood, J. D. (2022) · Journal of Emotional and Behavioral Disorders · doi:10.1177/10634266221099242 · Cited by 63 · see citing works
Effects of an Automated Vocabulary and Comprehension Intervention
Spencer, E. J., Goldstein, H., Sherman, A., Noe, S., Tabbah, R., Ziolkowski, R. A., & Schneider, N. (2012) · Journal of Early Intervention · doi:10.1177/1053815112471990 · Cited by 72 · see citing works
Improving Learning for All Students Through Equity-Based Inclusive Reform Practices
Choi, J. H., Meisenheimer, J., McCart, A. B., & Sailor, W. S. (2016) · Remedial and Special Education · doi:10.1177/0741932516644054 · Cited by 57 · see citing works
Sustainable bottled water: How nudging and Internet Search affect consumers’ choices
Grebitus, C., Roscoe, R. D., Loo, E. J. V., & Kula, I. (2020) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2020.121930 · Cited by 57 · see citing works
Stebner, F., Schuster, C., Weber, X.-L., Greiff, S., Leutner, D., & Wirth, J. (2022) · Metacognition and Learning · doi:10.1007/s11409-022-09322-x · Cited by 57 · see citing works
Generalizing about Public Health Interventions: A Mixed-Methods Approach to External Validity
Leviton, L. C. (2017) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031816-044509 · Cited by 59 · see citing works
The benefits of combining value for the self and others in utility-value interventions.
Priniski, S. J., Rosenzweig, E. Q., Canning, E. A., Hecht, C. A., Tibbetts, Y., Hyde, J. S., & Harackiewicz, J. M. (2019) · Journal of Educational Psychology · doi:10.1037/edu0000343 · Cited by 54 · see citing works
Goldstein, H., Kelley, E., Greenwood, C., McCune, L. A., Carta, J., Atwater, J., Guerrero, G., McCarthy, T. M., Schneider, N., & Spencer, T. (2016) · Journal of Speech, Language and Hearing Research · doi:10.1044/2015_jslhr-l-15-0227 · Cited by 44 · see citing works
Middle School Reading Comprehension and Content Learning Intervention for Below-Average Readers
Swanson, E. A., Wanzek, J., Vaughn, S. R., Fall, A., Roberts, G., Hall, C., & Miller, V. L. (2016) · Reading & Writing Quarterly · doi:10.1080/10573569.2015.1072068 · Cited by 53 · see citing works
Collaborative Strategic Reading: Findings From Experienced Implementers
Vaughn, S. R., Roberts, G., Klingner, J. K., Swanson, E. A., Boardman, A., Stillman‐Spisak, S. J., Mohammed, S. S., & Leroux, A. J. (2013) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2012.741661 · Cited by 52 · see citing works
Gaspard, H., Parrisius, C., Piesch, H., Kleinhansl, M., Wille, E., Nagengast, B., Trautwein, U., & Hulleman, C. S. (2021) · Journal of Educational Psychology · doi:10.1037/edu0000663 · Cited by 51 · see citing works
Self-citationCentury, J., Cassata, A., Rudnick, M., & Freeman, C. (2012) · The Journal of Behavioral Health Services & Research · doi:10.1007/s11414-012-9287-x · Cited by 51 · see citing works
The efficacy of Collaborative Strategic Reading in middle school science and social studies classes
Boardman, A. G., Klingner, J. K., Buckley, P. R., Annamma, S. A., & Lasser, C. J. (2015) · Reading and Writing · doi:10.1007/s11145-015-9570-3 · Cited by 48 · see citing works
Zvoch, K. (2012) · American Journal of Evaluation · doi:10.1177/1098214012452715 · Cited by 48 · see citing works
Getting a good start in school: Effects of INSIGHTS on children with high maintenance temperaments
McCormick, M. P., O’Connor, E. E., Cappella, E., & McClowry, S. (2014) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2014.10.006 · Cited by 49 · see citing works
Assessment in Early Childhood Science Education
Greenfield, D. B. (2015) · Springer eBooks · doi:10.1007/978-94-017-9505-0_16 · Cited by 48 · see citing works
Soicher, R. N., Becker‐Blease, K. A., & Bostwick, K. C. P. (2020) · Cognitive Research Principles and Implications · doi:10.1186/s41235-020-00255-0 · Cited by 52 · see citing works
Lewis, K. M., Bavarian, N., Snyder, F. J., Acock, A. C., Day, J., DuBois, D. L., Ji, P. Y., Schure, M. B., Silverthorn, N., Vuchinich, S., et al. (2012) · PubMed · Cited by 46 · see citing works
The effects of repetition on belief in naturalistic settings.
Fazio, L. K., Pillai, R. M., & Patel, D. (2022) · Journal of Experimental Psychology General · doi:10.1037/xge0001211 · Cited by 49 · see citing works
Hooshyar, D., Weng, X., Sillat, P. J., Tammets, K., Wang, M., & Hämäläinen, R. (2024) · Computers & Education · doi:10.1016/j.compedu.2024.105169 · Cited by 51 · see citing works
Wilcox, M. J., Gray, S. I., Guimond, A. B., & Lafferty, A. E. (2011) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2010.12.003 · Cited by 46 · see citing works
Behavioral Processes in Long-Lag Intervention Studies
Miller, D. T., Dannals, J. E., & Zlatev, J. J. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691616681645 · Cited by 45 · see citing works
Moving an Evidence-Based Parenting Program Into the Community
Roben, C. K. P., Dozier, M., Caron, E., & Bernard, K. (2017) · Child Development · doi:10.1111/cdev.12898 · Cited by 45 · see citing works
Caron, E., & Dozier, M. (2019) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-019-00924-3 · Cited by 43 · see citing works
Teachers can do it: Scalable identity-based motivation intervention in the classroom
Horowitz, E., Sorensen, N., Yoder, N. P., & Oyserman, D. (2018) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2018.04.004 · Cited by 43 · see citing works
The Impact of the Measures of Academic Progress (MAP) Program on Student Reading Achievement
Cordray, D. S., Pion, G. M., Brandt, C., & Molefe, A. C. (2013) · Society for Research on Educational Effectiveness · Cited by 43 · see citing works
Self-citationThe Targeted Reading Intervention: Face–to–Face vs. Webcam Literacy Coaching of Classroom Teachers
Vernon‐Feagans, L., Bratsch‐Hines, M., Varghese, C. A., Bean, A., & Hedrick, A. (2015) · Learning Disabilities Research and Practice · doi:10.1111/ldrp.12062 · Cited by 42 · see citing works
O'Connor, E., Rodriguez, E. T., Cappella, E., Morris, J., & McClowry, S. (2012) · Journal of Community Psychology · doi:10.1002/jcop.21482 · Cited by 41 · see citing works
Enhancing the Academic Development of Shy Children: A Test of the Efficacy of INSIGHTS
O’Connor, E. E., Cappella, E., McCormick, M. P., & McClowry, S. (2014) · School Psychology Review · doi:10.1080/02796015.2014.12087426 · Cited by 40 · see citing works
Promises and Pitfalls of Adapting Utility Value Interventions for Online Math Courses
Rosenzweig, E. Q., Hulleman, C. S., Barron, K. E., Kosovich, J. J., Priniski, S. J., & Wigfield, A. (2019) · The Journal of Experimental Education · doi:10.1080/00220973.2018.1496059 · Cited by 41 · see citing works
Self-citationHemmeter, M. L., Snyder, P., & Fox, L. (2017) · School Mental Health · doi:10.1007/s12310-017-9239-y · Cited by 38 · see citing works
Desimone, L. M., Wolford, T. E., & Hill, K. L. (2016) · AERA Open · doi:10.1177/2332858416679599 · Cited by 37 · see citing works
Reading Recovery: An Evaluation of the Four-Year i3 Scale-Up
May, H., Sirinides, P., Gray, A. M., & Goldsworthy, H. (2016) · doi:10.12698/cpre.2016.readingrecovery · Cited by 37 · see citing works
Whittaker, J. V., Kinzie, M. B., Vitiello, V. E., DeCoster, J., Mulcahy, C., & Barton, E. A. (2020) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2019.1710884 · Cited by 36 · see citing works
Herbein, E., Golle, J., Tibus, M., Zettler, I., & Trautwein, U. (2018) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2018.09.003 · Cited by 45 · see citing works
Boardman, A. G., Buckley, P. R., Vaughn, S. R., Roberts, G., Scornavacco, K., & Klingner, J. K. (2016) · Journal of Learning Disabilities · doi:10.1177/0022219416640784 · Cited by 35 · see citing works
Interaction of Theory and Practice to Assess External Validity
Leviton, L. C., & Trujillo, M. D. (2017) · Evaluation Review · doi:10.1177/0193841x15625289 · Cited by 35 · see citing works
McCormick, M. P., Neuhaus, R., O’Connor, E. E., White, H. I., Horn, E. P., Harding, S., Cappella, E., & McClowry, S. (2020) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2020.1831117 · Cited by 34 · see citing works
Implementing Team-Based Learning in Middle School Social Studies Classes
Wanzek, J., Kent, S. C., Vaughn, S. R., Swanson, E. A., Roberts, G., & Haynes, M. (2015) · The Journal of Educational Research · doi:10.1080/00220671.2014.893224 · Cited by 34 · see citing works
Pas, E. T., Johnson, S. R., Debnam, K. J., Hulleman, C. S., & Bradshaw, C. P. (2019) · Remedial and Special Education · doi:10.1177/0741932518805192 · Cited by 34 · see citing works
Self-citationHemmeter, M. L., Barton, E. E., Fox, L., Vatland, C. H., Henry, G., Pham, L. D., Horth, K., Taylor, A. L., Binder, D. P., von der Embse, M., et al. (2021) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2021.10.003 · Cited by 33 · see citing works
McCormick, M. P., Weiland, C., Hsueh, J., Maier, M. F., Hagos, R. M., Snow, C. E., Leacock, N., & Schick, L. (2019) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2019.06.012 · Cited by 32 · see citing works
Piasta, S. B., Justice, L. M., McGinty, A. S., Mashburn, A. J., & Slocum, L. (2015) · Child & Youth Care Forum · doi:10.1007/s10566-015-9305-2 · Cited by 32 · see citing works
LoCasale‐Crouch, J., DeCoster, J., Cabell, S. Q., Pianta, R. C., Hamre, B. K., Downer, J. T., Hatfield, B. E., Larsen, R., Burchinal, M., Howes, C. H., et al. (2016) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2015.12.022 · Cited by 31 · see citing works
Gerstner, J. J., & Finney, S. J. (2013) · Research & Practice in Assessment · Cited by 31 · see citing works
Oyserman, D., O’Donnell, S. C., Sorensen, N., & Wingert, K. M. (2021) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2021.101993 · Cited by 30 · see citing works
Fidelity in After-School Program Intervention Research: A Systematic Review
Maynard, B. R., Peters, K. E., Vaughn, M. G., & Sarteschi, C. M. (2013) · Research on Social Work Practice · doi:10.1177/1049731513491150 · Cited by 30 · see citing works
Berg, J., Bradshaw, C. P., Jo, B., & Ialongo, N. S. (2016) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-016-0738-1 · Cited by 30 · see citing works
Rimm‐Kaufman, S. E., Merritt, E. G., Lapan, C., DeCoster, J., Hunt, A., & Bowers, N. (2021) · Journal of Applied Developmental Psychology · doi:10.1016/j.appdev.2020.101236 · Cited by 31 · see citing works
Kutash, K., Cross, B. L., Madias, A., Duchnowski, A. J., & Green, A. L. (2012) · Journal of Child and Family Studies · doi:10.1007/s10826-012-9565-5 · Cited by 28 · see citing works
- Authors
- Hulleman, C. S., & Cordray, D. S.
- Affiliations
- Vanderbilt University (US)
- ORCID
- Hulleman, Chris S.
- Venue
- Journal of Research on Educational Effectiveness
- Publisher
- Informa UK Limited
- Published
- 2009-01-14
- Volume
- 2
- Issue
- 1
- Pages
- 88-110
- ISSN
- 1934-5739 (ISSN-L)
- 1934-5747
- OpenAlex
- W2088086259
- Cited by
- 291
- References
- 29
- Citations by year
- 2026: 4 · 2025: 10 · 2024: 10 · 2023: 15 · 2022: 17 · 2021: 28 · 2020: 25 · 2019: 22 · 2018: 20 · 2017: 24 · 2016: 24 · 2015: 24 · 2014: 21 · 2013: 17 · 2012: 14
- FWCI
- 17.58
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Research on Educational Effectiveness (publishedVersion)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Education › Educational Research and Methods
- Keywords
- intervention fidelity, motivational interventions, student motivation, classroom implementation, teacher factors, multilevel analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
