Implementation and effects of a school-wide data-based decision making intervention
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Background.Despite growing international interest in the use of data to improve education, few studies examining the effects on student achievement are yet available.Purpose.In the present study, the effects of a two-year data-based decision making intervention on student achievement growth were investigated.Method.Fifty-three primary schools participated in the project, and student achievement data were collected over the two years before and two years during the intervention.Linear mixed models were used to analyze the differential effect of data use on student achievement. Findings.A positive mean intervention effect was estimated, with an average effect of approximately one extra month of schooling.Furthermore, the results suggest that the intervention especially significantly improved the performances of students in low-SES schools. Interaction EffectsIn Model 5, interaction effects were introduced for school-SES and the intervention, and for student-SES and the intervention.It was hypothesized that the intervention effect would be larger for schools with a large population of students with low-SES (hypothesis 3).The findings of this study support this hypothesis: the interaction effect for school-SES and intervention was positive but not significant for schools with low-SES, and negative but not significant for high-SES schools.Hypothesis 4 concerned the interaction between intervention effect and student-SES.It was stated that the intervention effect would be larger for low-SES students.The interaction effect for intervention and low-SES student was significant and positive (effect of 1.05), but surprisingly the effect found for high-SES students and the intervention was also positive and significant (also an effect of 1.05).Given that the interaction effect for student-SES and intervention is conditional for the interaction effect of school-SES and intervention, it is interesting to compare effects for the combination of student and school-SES even though the latter were not significant.These combinations indicate that the effect of the intervention will only lead to a negative effect on student achievement for medium-SES students in high-SES schools, and is positive and quite large for low-and high-SES students, regardless of their school's average SES.It was assumed that school and student background characteristics would not influence the effect of intervention, and it is interesting to monitor the stability of the estimated intervention effect across models.The estimated positive main intervention effect and the random effect variability over schools is stable across Models 1 to 4. Furthermore, the random intervention effect could not be attributed to differences in background information, or to differential growth in student and school achievement.Therefore, it is concluded that the random intervention effects are identified based on differences in achievement of students in the prior-to-intervention group and during-intervention group.The main intervention effect only decreased after including interaction effects in Model 5.These findings provide partial support for hypothesis 1 (the implementation of DBDM will lead to higher student achievement, there is a positive intervention effect).Although the model fit results are not presented, the multilevel models were investigated with respect to model fit.For each model, a residual analysis was carried out to identify outliers and to investigate distributional assumptions of the residuals.The level-1 residuals showed nine outliers, which most likely stemmed from incorrectly entered observations in the student monitoring system.Some of the scores did not fall within the possible score range of the administered test.Some bias in the fitted values were detected because of observed scores of zero.Since the scores were assumed to be normally distributed and by ignoring this lower-bound, some negative fitted scores were obtained.This lowerbound problem was ignored since the few negative predicted scores did not influence the parameter estimates, which were based on a total of 66,486 observed scores.The student random effects were approximately normally distributed, but the random student effect distribution for grade 6-8 scores was very peaked.The variability in random latent student scores in grade 6-8 was less than expected since the correlation between random effects (grade 3-5 and grade 6-8) was high.The variance in effects across students was relatively low, leading to a more peaked distribution of the random effect distribution of student scores in grade 6-8.However, this random effect was needed to properly represent the scores of students in grade 6 to 8 despite the high correlation with the scores grade 3 to 5. 3The Effects of a Schoolwide Data-Based Decision Making Intervention on Elementary School's Student Achievement Growth for Mathematics and Spelling
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905
Learner-Centered Teacher-Student Relationships Are Effective: A Meta-Analysis
Cornelius-White, J. (2007) · Review of Educational Research · doi:10.3102/003465430298563 · Cited by 1,812
Desimone, L. M., Porter, A. C., Garet, M. S., Yoon, K. S., & Birman, B. F. (2002) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737024002081 · Cited by 1,688
Social Networks and Organizations
Kilduff, M., & Tsai, W. (2003) · doi:10.4135/9781849209915 · Cited by 1,581
Sung, Y.-T., Chang, K.-E., & Liu, T.-C. (2016) · Computers & Education · doi:10.1016/j.compedu.2015.11.008 · Cited by 1,574
Formative assessment: a critical review
Bennett, R. E. (2011) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969594x.2010.513678 · Cited by 1,498 · cited 4×
“…DBDM requires knowledge and skills with regard to the components of DBDM: 1) analyzing an interpreting data (also known as ‘data literacy’) (Bennett, 2011; Mandinach, Honey, & Light, 2006); 2) setting SMART and challenging goals (Visscher & Ehren, 2011); 3) choosing effective strategies…”
BackgroundMethodInfluentialShall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences
Schmidt, S. (2009) · Review of General Psychology · doi:10.1037/a0015108 · Cited by 1,125 · cited 2×
“spelling) and to strengthen the generalizability of the findings concerning mathematics, we conducted a conceptual replication (Makel & Plucker, 2014; Schmidt, 2009) of the 2011 study.”
BackgroundResultFacts Are More Important Than Novelty: Replication in the Education Sciences
Makel, M. C., & Plucker, J. A. (2014) · Educational Researcher · doi:10.3102/0013189x14545513 · Cited by 609
How Methodological Features Affect Effect Sizes in Education
Cheung, A. C. K., & Slavin, R. E. (2016) · Educational Researcher · doi:10.3102/0013189x16656615 · Cited by 529
A Systemic View of Implementing Data Literacy in Educator Preparation
Mandinach, E. B., & Gummer, E. S. (2013) · Educational Researcher · doi:10.3102/0013189x12459803 · Cited by 492
Cited by
Kippers, W. B., Wolterinck, C. H. D., Schildkamp, K., Poortman, C. L., & Visscher, A. J. (2018) · Teaching and Teacher Education · doi:10.1016/j.tate.2018.06.015 · Cited by 119
On the value of data-based decision making in education: The evidence from six intervention studies
Visscher, A. J. (2021) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2020.100899 · Cited by 61
Why a Data‐Based Decision‐Making Intervention Works in Some Schools and Not in Others
Keuning, T., Van Geel, M., & Visscher, A. (2017) · Learning Disabilities Research & Practice · doi:10.1111/ldrp.12124 · Cited by 60
Effects of a data-based decision making intervention on student achievement
Staman, L. (., Timmermans, A. C. (., & Visscher, A. J. (. (2017) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2017.07.002 · Cited by 38
Metadata
- Authors
- Keuning&Geel, T. &.
- Published
- 2016-04-07
- OpenAlex
- W2511106946
- Cited by
- 7
- References
- 28
- Type
- dissertation
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Implementation and effects of a school-wide data-based decision making intervention. https://app.cochat.ai/papers/10.3990/1.9789036541190
- The paper
- Keuning&Geel, T. &. (2016). Implementation and effects of a school-wide data-based decision making intervention. https://doi.org/10.3990/1.9789036541190
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.