Implementing Effective Educational Practices at Scales of Social Importance
The DOI matches its Crossref record; 62 of the 65 references it cites were found in a registry or an index.
Abstract
Implementing evidence-based practices is becoming both a goal and standard across medicine, psychology, and education. Initial successes, however, are now leading to questions about how successful demonstrations may be expanded to scales of social importance. In this paper, we review lessons learned about scaling up evidence-based practices gleaned from our experience implementing school-wide positive behavioral interventions and supports (PBIS) across more than 23,000 schools in the USA. We draw heavily from the work of Flay et al. (Prev Sci 6:151-175, 2005. doi: 10.1007/s11121-005-5553-y ) related to defining evidence-based practices, the significant contributions from the emerging "implementation science" movement (Fixsen et al. in Implementation research: a synthesis of the literature, University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231), Tampa 2005), and guidance we have received from teachers, family members, students, and administrators who have adopted PBIS.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Positive psychology: An introduction.
Seligman, M. E. P., & Csikszentmihalyi, M. (2000) · American Psychologist · doi:10.1037/0003-066x.55.1.5 · Cited by 11,351 · cited 1×
“This positive and preventive approach helps to build thriving classroom and school cultures and resilience among individuals and groups in schools and society (Seligman and Csikszentmihalyi 2000).”
BackgroundKilzer, E., & Skinner, B. F. (1953) · The American Catholic Sociological Review · doi:10.2307/3707860 · Cited by 8,943
Symonds, P. M. (1954) · Journal of Educational Psychology · doi:10.1037/h0051901 · Cited by 4,875
(2005) · Cited by 3,748
Self-citationSkinner, B. F. (2022) · ABA España eBooks · doi:10.26741/978-84-09-40105-5 · Cited by 2,717
American Journal of Mental Retardation
(2014) · Encyclopedia of Special Education · doi:10.1002/9781118660584.ese0112 · Cited by 1,831
Single-Case Designs Technical Documentation.
Kratochwill, T. R., Hitchcock, J. H., Horner, R., Levin, J., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010) · Cited by 1,345
Self-citationStandards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination
Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., Mościcki, E. K., Schinke, S., Valentine, J. C., & Ji, P. (2005) · Prevention Science · doi:10.1007/s11121-005-5553-y · Cited by 1,338
Research in Special Education: Scientific Methods and Evidence-Based Practices
Odom, S. L., Brantlinger, E., Gersten, R., Horner, R. H., Thompson, B., & Harris, K. R. (2005) · Exceptional Children · doi:10.1177/001440290507100201 · Cited by 970
Self-citationApplied Behavior Analysis for Teachers
Alberto, P., Troutman, A., Axe, J. B., & Simmons University, B. (2024) · ABA España eBooks · doi:10.26741/978-84-09-59866-3 · Cited by 956
Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010) · Journal of Positive Behavior Interventions · doi:10.1177/1098300709334798 · Cited by 868
Horner, R. H., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A. W., & Esperanza, J. (2009) · Journal of Positive Behavior Interventions · doi:10.1177/1098300709332067 · Cited by 826
Self-citationApplied Behavior Analysis for Teachers
Alberto, P., Troutman, A., Axe, J. B., & Simmons University, B. (2024) · ABA España eBooks · doi:10.26741/978-84-09-59867-0 · Cited by 767
Lau, A. S. (2006) · Clinical Psychology: Science and Practice · doi:10.1111/j.1468-2850.2006.00042.x · Cited by 624 · cited 1×
“implementation fidelity, and enhanced outcomes across the range of student characteristics (Lau 2006).”
BackgroundExamining the Evidence Base for School-Wide Positive Behavior Support
Horner, R. H., Sugai, G., & Anderson, C. M. (2010) · Focus on Exceptional Children · doi:10.17161/foec.v42i8.6906 · Cited by 581
Self-citationReporting standards for research in psychology: why do we need them? What might they be?
APA Publications and Communications Board Working Group on Journal Article Reporting Standards (2008) · American Psychologist · doi:10.1037/0003-066x.63.9.839 · Cited by 571
Horner, R. H., Todd, A. W., Lewis-Palmer, T., Irvin, L. K., Sugai, G., & Boland, J. B. (2004) · Journal of Positive Behavior Interventions · doi:10.1177/10983007040060010201 · Cited by 560
Self-citationAssessment of Treatment Integrity in School Consultation and Prereferral Intervention
Gresham, F. M. (1989) · School Psychology Review · doi:10.1080/02796015.1989.12085399 · Cited by 558
Statewide Implementation of Evidence-Based Programs
Fixsen, D., Blase, K., Metz, A., & Van Dyke, M. (2013) · Exceptional Children · doi:10.1177/001440291307900206 · Cited by 535
Self-citationEffective Behavior Support: A Systems Approach to Proactive Schoolwide Management
Lewis, T. J., & Sugai, G. (2017) · Focus on Exceptional Children · doi:10.17161/foec.v31i6.6767 · Cited by 476
Self-citationBradshaw, C. P., Koth, C. W., Thornton, L. A., & Leaf, P. J. (2009) · Prevention Science · doi:10.1007/s11121-008-0114-9 · Cited by 448
Positive Behavior Support for People with Developmental Disabilities: A Research Synthesis
Carr, E. G., Horner, R. H., Turnbull, A. P., Marquis, J. G., McLaughlin, D. M., McAtee, M. L., Smith, C. E., Ryan, K. A., Ruef, M. B., Doolabh, A., et al. (1999) · Medical Entomology and Zoology · Cited by 412
Self-citationEvidence-based Kernels: Fundamental Units of Behavioral Influence
Embry, D. D., & Biglan, A. (2008) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-008-0036-x · Cited by 396 · cited 1×
“Operationally defined core features are the observable and countable elements of a practice that are directly related to improving targeted outcomes (Blase and Fixsen 2013; Embry 2004; Embry and Biglan 2008).”
BackgroundBradshaw, C. P., Koth, C. W., Bevans, K. B., Ialongo, N., & Leaf, P. J. (2008) · School Psychology Quarterly · doi:10.1037/a0012883 · Cited by 389
Building Positive Behavior Support Systems in Schools: Functional Behavioral Assessment
Crone, D. A., & Horner, R. H. (2003) · Cited by 371
Self-citationPREVENTING ANTISOCIAL BEHAVIOR IN THE SCHOOLS
Mayer, G. R. (1995) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1995.28-467 · Cited by 340
Ingram, K., Lewis-Palmer, T., & Sugai, G. (2005) · Journal of Positive Behavior Interventions · doi:10.1177/10983007050070040401 · Cited by 335
Self-citationFirst Step to Success: An Early Intervention Approach for Preventing School Antisocial Behavior
WALKER, H. M., KAVANAGH, K., STILLER, B., GOLLY, A., SEVERSON, H. H., & FEIL, E. G. (1998) · Journal of Emotional and Behavioral Disorders · doi:10.1177/106342669800600201 · Cited by 312
Didden, R., Duker, P. C., & Korzilius, H. P. L. M. (1997) · PubMed · Cited by 307
Irvin, L. K., Tobin, T. J., Sprague, J. R., Sugai, G., & Vincent, C. G. (2004) · Journal of Positive Behavior Interventions · doi:10.1177/10983007040060030201 · Cited by 302 · cited 1×
“ODRs, however, can be affected by many variables beyond change in student behavior (Irvin et al. 2004).”
BackgroundSelf-citationKlingner, J. K., Artiles, A. J., Kozleski, E., Harry, B., Zion, S., Tate, W., Zamora Durán, G., & Riley, D. (2005) · Education Policy Analysis Archives · doi:10.14507/epaa.v13n38.2005 · Cited by 287
Current Status and Future Directions of School-Based Behavioral Interventions
Gresham, F. M. (2004) · School Psychology Review · doi:10.1080/02796015.2004.12086252 · Cited by 277
Waasdorp, T. E. (2012) · Archives of Pediatrics & Adolescent Medicine · doi:10.1001/archpediatrics.2011.755 · Cited by 263
Bradshaw, C. P., Reinke, W. M., Brown, L. D., Bevans, K. B., & Leaf, P. J. (2008) · Education and Treatment of Children · doi:10.1353/etc.0.0025 · Cited by 251
Defining and Describing Schoolwide Positive Behavior Support
Sugai, G., & Horner, R. H. (2009) · Issues in Clinical Child Psychology · doi:10.1007/978-0-387-09632-2_13 · Cited by 224
Self-citationA Contextual Consideration of Culture and School-Wide Positive Behavior Support
Sugai, G., O’Keeffe, B. V., & Fallon, L. M. (2012) · Journal of Positive Behavior Interventions · doi:10.1177/1098300711426334 · Cited by 204
Self-citationChanging Cultural Practices: A Contextualist Framework for Intervention Research
Biglan, A. (1995) · Cited by 193
Implementation Science: A Potential Catalyst for Delivery System Reform
Fisher, E. S., Shortell, S. M., & Savitz, L. A. (2016) · JAMA · doi:10.1001/jama.2015.17949 · Cited by 190
Factors Related to Sustained Implementation of Schoolwide Positive Behavior Support.
McIntosh, K., Mercer, S. H., Hume, A. E., Frank, J. L., Turri, M. G., & Mathews, S. (2013) · Exceptional Children · Cited by 182
Maggin, D. M., Zurheide, J., Pickett, K. C., & Baillie, S. J. (2015) · Journal of Positive Behavior Interventions · doi:10.1177/1098300715573630 · Cited by 166
Fallon, L. M., O’Keeffe, B. V., & Sugai, G. M. (2012) · Journal of Positive Behavior Interventions · doi:10.1177/1098300712442242 · Cited by 163
Self-citationChilds, K. E., Kincaid, D., George, H. P., & Gage, N. A. (2016) · Journal of Positive Behavior Interventions · doi:10.1177/1098300715590398 · Cited by 140
Building Positive Behavior Support Systems in Schools: Functional Behavioral Assessment.
Bagnell, A. L., & Bostic, J. Q. (2004) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/01.chi.0000132828.82347.70 · Cited by 133
Horner, R. H., Kincaid, D., Sugai, G., Lewis, T., Eber, L., Barrett, S., Dickey, C. R., Richter, M., Sullivan, E., Boezio, C., et al. (2014) · Journal of Positive Behavior Interventions · doi:10.1177/1098300713503685 · Cited by 121
Self-citationTreatment integrity: A foundation for evidence-based practice in applied psychology.
(2014) · American Psychological Association eBooks · doi:10.1037/14275-000 · Cited by 120
Walker, H. M., Seeley, J. R., Small, J., Severson, H. H., Graham, B. A., Feil, E. G., Serna, L., Golly, A. M., & Forness, S. R. (2009) · Journal of Emotional and Behavioral Disorders · doi:10.1177/1063426609341645 · Cited by 117
Community‐based prevention using simple, low‐cost, evidence‐based kernels and behavior vaccines
Embry, D. D. (2004) · Journal of Community Psychology · doi:10.1002/jcop.20020 · Cited by 104
PBIS in alternative education settings: positive support for youth with high-risk behavior
Simonsen, B., & Sugai, G. (2013) · Education and Treatment of Children · doi:10.1353/etc.2013.0030 · Cited by 104 · cited 1×
“A growing research base affirms that Tier I PBIS is a set of defined core features that are functionally related to improved social behavior for students in elementary, middle, and high schools (Bradshaw et al. 2008b, 2010; Flannery et al. 2013; Horner et al. 2010; Simonsen and Sugai 2013).”
BackgroundSelf-citationTechnical Adequacy of the SWPBIS Tiered Fidelity Inventory
McIntosh, K., Massar, M. M., Algozzine, R. F., George, H. P., Horner, R. H., Lewis, T. J., & Swain-Bradway, J. (2017) · Journal of Positive Behavior Interventions · doi:10.1177/1098300716637193 · Cited by 104
Self-citationKelm, J. L., McIntosh, K., & Cooley, S. (2014) · Canadian Journal of School Psychology · doi:10.1177/0829573514540266 · Cited by 74
The need for a science of larger social units: A contextual approach
Biglan, A., Glasgow, R. E., & Singer, G. (1990) · Behavior Therapy · doi:10.1016/s0005-7894(05)80277-1 · Cited by 62
Effects of Team-Initiated Problem Solving on Decision Making by Schoolwide Behavior Support Teams
Todd, A. W., Horner, R. H., Newton, J. S., Algozzine, R. F., Algozzine, K. M., & Frank, J. L. (2011) · Journal of Applied School Psychology · doi:10.1080/15377903.2011.540510 · Cited by 62
Self-citationMcIntosh, K., Kim, J., Mercer, S. H., Strickland-Cohen, M. K., & Horner, R. H. (2015) · Assessment for Effective Intervention · doi:10.1177/1534508414556503 · Cited by 57
Self-citationFlannery, K. B., Frank, J. L., Kato, M. M., Doren, B., & Fenning, P. (2013) · The High School Journal · doi:10.1353/hsj.2013.0015 · Cited by 51
Jones, C. H., Caravaca, L., Cizek, S., Horner, R. H., & Vincent, C. G. (2006) · Multiple Voices for Ethnically Diverse Exceptional Learners · doi:10.56829/muvo.9.1.0311x7477113q741 · Cited by 45
Self-citationUsing a Problem-Solving Model to Enhance Data-Based Decision Making in Schools
Newton, S. J., Horner, R. H., Algozzine, R. F., Todd, A. W., & Algozzine, K. M. (2008) · Issues in clinical child psychology · doi:10.1007/978-0-387-09632-2_23 · Cited by 38
Self-citationJones, C. R., Caravaca, L., Cizek, S., Horner, R. H., & Vincent, C. G. (2010) · Multiple Voices for Ethnically Diverse Exceptional Learners · doi:10.5555/muvo.9.1.0311x7477113q741 · Cited by 34
Self-citationPositive Behavior Support, Sustainability and Continuous Regeneration
McIntosh, K., & Turri, M. G. (2014) · Wiley eBooks · doi:10.1002/9781118660584.ese1903 · Cited by 17
Using Implementation Science to Translate Foundation Strategy
Metz, A. J. R., & Easterling, D. V. (2016) · The Foundation Review · doi:10.9707/1944-5660.1302 · Cited by 16
Moreno, G., Wong-Lo, M., & Bullock, L. M. (2013) · Preventing School Failure Alternative Education for Children and Youth · doi:10.1080/1045988x.2012.763156 · Cited by 14
Killeen, K. M., & Sipple, J. W. (2005) · Education Policy Analysis Archives · doi:10.14507/epaa.v13n19.2005 · Cited by 13
Parallel Block Scheduling: An Alternative Structure.
Canady, R. L., & Reina, J. M. (1993) · Principal · Cited by 11
Research works printed but not linked: 10
- Biglan, A., Cody, C., Aldridge, W. A., II, Dabroski, A., & Kjellstrand, J. (2011). The promise neighborhoods initiative: Improving developmental outcomes through comprehensive interventions. The Journal of the Institute for Comprehensive Community Development, 2, 25–34.
- Blase, K., & Fixsen, D. (2013). Core intervention components: Identifying and operationalizing what makes programs work. Washington, DC: ASPE Research Brief, Office of the Assistant Secretary for Planning and Evaluation, Office of Human Services Policy, U.S. Department of Health and Human Services.
- Crone, D., Hawken, L., & Horner, R. H. (2010). Responding to problem behavior in schools: The behavior education program (2nd edn). The Guilford Practical Intervention in the Schools Series. New York, NY: Guilford Press.
- Fixsen, D. L., Blase, K. A., Horner, R. H., & Sugai, G. (2010). An overview of the state implementation and scaling up of evidence-based practices center (SISEP): Scaling up brief, 4. Chapel Hill, NC: FPG Child Development Institute, University of North Carolina Chapel Hill.
- Latham, G. I. (1992). Interacting with at-risk children: The positive approach. Principal, 72, 26–30.
- Meyer, L. H., & Evans, I. M. (1989). Non aversive intervention for behavior problems. Baltimore, MD: Paul H. Brookes.
- O’Neill, R. E., Albin, R. W., Storey, K., Horner, R. H., & Sprague, J. R. (Eds.). (2015). Functional assessment and program development for problem behavior: A practical handbook, instructor’s edition (3rd ed.). Stamford, CT: Cengage Learning.
- Sprick, R., Sprick, M., & Garrison, M. (1992). Foundations: Developing positive school-wide discipline policies. Longmont, CO: Sopris West.
- Sugai, G., & Horner, R. H. (1999). Discipline and behavioral support: Preferred processes and practices. Effective School Practices, 17, 10–22.
- Tobin, T. J., Vincent, C. G., Horner, R. H., Rossetto Dickey, C., & May, S. A. (2012). Fidelity measures to improve implementation of behavioural support. International Journal of Positive Behavioural Support, 2, 12–19.
Cited by
Cook, C. R., Lyon, A. R., Locke, J., Waltz, T., & Powell, B. J. (2019) · Prevention Science · doi:10.1007/s11121-019-01017-1 · Cited by 336
“Develop local policy that supports implementation was added based on research findings related to universal prevention efforts, such as schoolwide positive behavior intervention supports, suggesting that changes to school discipline policy lead to changes in adult behavior regarding how educators effectively respond t…”
BackgroundBuilding Schools’ Readiness to Implement a Comprehensive Approach to School Safety
Kingston, B. E., Mattson, S. A., Dymnicki, A. B., Spier, E., Fitzgerald, M. M., Shipman, K., Goodrum, S., Woodward, W., Witt, J., Hemphill, S. A., et al. (2018) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-018-0264-7 · Cited by 71
Fox, R. A., Leif, E., Moore, D. W., Furlonger, B., Anderson, A., & Sharma, U. (2021) · Education and Treatment of Children · doi:10.1007/s43494-021-00056-0 · Cited by 66
“Sustained Implementation of SWPBIS Although the SWPBIS framework has been widely adopted, sustained implementation has emerged as a critical issue (Horner et al., 2017).”
MethodSanchez, Z. M., Valente, J. Y., Galvão, P. P. D. O., Gubert, F. D. A., Melo, M. H. D. S., Caetano, S. C., Mari, J. D. J., & Cogo‐Moreira, H. (2021) · Addiction · doi:10.1111/add.15358 · Cited by 59
Edyburn, K. L., Bertone, A., Raines, T. C., Hinton, T., Twyford, J. M., & Dowdy, E. T. (2022) · School Psychology Review · doi:10.1080/2372966x.2021.2024767 · Cited by 57
Metadata
- Authors
- Horner, R. H., Sugai, G., & Fixsen, D. L.
- Affiliations
- University of Oregon (US)
- University of Connecticut (US)
- University of North Carolina at Chapel Hill (US)
- Corresponding author
- Horner, Robert H.
- Venue
- Clinical Child and Family Psychology Review
- Published
- 2017-01-23
- Volume
- 20
- Issue
- 1
- Pages
- 25-35
- ISSN
- 1096-4037 (ISSN-L)
- 1573-2827
- OpenAlex
- W2581310436
- Semantic Scholar
- 7156d0e3b978d5fe67763d6e2d263aab97c6e104
- Cited by
- 114
- References
- 65
- Citations by year
- 2026: 9 · 2025: 16 · 2024: 12 · 2023: 14 · 2022: 11 · 2021: 18 · 2020: 17 · 2019: 6 · 2018: 8 · 2017: 3
- FWCI
- 1.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Clinical Child and Family Psychology Review (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Office of Special Education Programs, Office of Special Education and Rehabilitative Services (H326S98003, H326K120005)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › School Health and Nursing Education
- Keywords
- Positive Behavioral Interventions and Supports, implementation science, student discipline, implementation fidelity, lessons learned
- MeSH terms
- Behavior Therapy, Child, Child Behavior, Humans, Schools, Social Behavior, Evidence-Based Practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.