Factors Influencing the Functioning of Data Teams
Open accessAbstract
Background Data-based decision making can lead to increased student achievement; however, schools struggle with the implementation of data-based decision making. Professional development in the use of data is therefore urgently needed. However, professional development is often ineffective in terms of improving the knowledge, skills, and attitude of the receiver. Purpose We need a more fundamental understanding of how we can increase the effectiveness of data-use-related professional development. This study therefore focuses on the factors influencing a professional development intervention for data-based decision making: the data team procedure. Data teams are teams of teachers and school leaders who collaboratively learn how to use data, following a structured approach and guided by a facilitator from the university. Based on an extensive literature review, we developed a data use framework in which the use of data is influenced by data characteristics, school organization characteristics, and user and team characteristics. Research Design We conducted case studies. Data Collection We focused on observing in depth the factors that influence the work of the data teams and interviewing the data team members about these factors. Four data teams of six schools for upper secondary education were followed over a period of 2 years. We observed and analyzed 34 meetings and analyzed 23 interviews, combined with our field notes. Although this pilot study only permits analytical generalization of the findings, the findings provide more in-depth insight into the factors that enable and hinder interventions, focusing on supporting collaborative data use in schools. Findings The results show that several data characteristics (access and availability of high-quality data), school organizational characteristics (a shared goal, leadership, training and support, involvement of relevant stakeholders), and individual and team characteristics (data literacy, pedagogical content knowledge [PCK], organizational knowledge, attitude, and collaboration) influence the use of data in data teams. The results also show that these influencing factors are interrelated. Conclusions Schools need support in all aspects of the use of data (from formulation of a problem definition to taking action based on the data). This study can form a starting point for larger studies into the factors influencing these types of professional development interventions to ensure effective implementation and sustainability.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Cites
(1984) · Cited by 81,983 · see citing works
Case Study Research: Design and Methods
Chaiklin, H. (1991) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-199102000-00025 · Cited by 39,631 · see citing works
Situated Learning: Legitimate Peripheral Participation.
Bloch, M. E. F., Lave, J., & Wenger, É. (1994) · Man · doi:10.2307/2804509 · Cited by 39,323 · see citing works
Lave, J., & Wenger, E. (1991) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815355 · Cited by 33,219 · see citing works
Those Who Understand: Knowledge Growth in Teaching
SHULMAN, L. S. (1986) · Educational Researcher · doi:10.3102/0013189x015002004 · Cited by 18,546 · see citing works
Assessment and Classroom Learning
Black, P., & Wiliam, D. (1998) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969595980050102 · Cited by 8,152 · see citing works
Desimone, L. M. (2009) · Educational Researcher · doi:10.3102/0013189x08331140 · Cited by 4,862 · see citing works
A Purposeful Approach to the Constant Comparative Method in the Analysis of Qualitative Interviews
Boeije, H. (2002) · Quality and Quantity · doi:10.1023/a:1020909529486 · Cited by 3,073 · see citing works
COGNITIVE APPRENTICESHIP: MAKING THINKING VISIBLE
Collins, A., Brown, J. S., & Holum, A. (1991) · Cited by 1,516 · see citing works
Horn, I. S., & Little, J. W. (2010) · American Educational Research Journal · doi:10.3102/0002831209345158 · Cited by 762 · see citing works
A Systemic View of Implementing Data Literacy in Educator Preparation
Mandinach, E. B., & Gummer, E. S. (2013) · Educational Researcher · doi:10.3102/0013189x12459803 · Cited by 492 · see citing works
Professional development in education
Webster, V., Franey, J., & Webster, A. (1994) · Bristol Research (University of Bristol) · Cited by 477 · see citing works
A Perfect Time for Data Use: Using Data-Driven Decision Making to Inform Practice
Mandinach, E. B. (2012) · Educational Psychologist · doi:10.1080/00461520.2012.667064 · Cited by 468 · see citing works
Teaching and Teacher Education
Cochran‐Smith, M. (2016) · Educational Researcher · doi:10.3102/0013189x16639040 · Cited by 440 · see citing works
Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors
Schildkamp, K., & Kuiper, W. (2010) · Teaching and Teacher Education · doi:10.1016/j.tate.2009.06.007 · Cited by 406 · cited 13× · see citing works
“…the literature: Access to high quality data (Coburn & Turner, 2011; Kerr et al., 2006; Mandinach & Honey, 2008; Marsh et al., 2006; Means, et al., 2010; Schildkamp & Kuiper, 2010; Sharkey & Murnane, 2006; Wayman et al., 2006; Wayman & Stringfield 2006; Wayman, et al., 2007; Wayman et al., 2012).”
BackgroundMethodInfluentialSelf-citationAccountability Policies and Teacher Decision Making: Barriers to the Use of Data to Improve Practice
Ingram, D., Louis, K. S., & Schroeder, R. G. (2004) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1111/j.1467-9620.2004.00379.x · Cited by 373 · see citing works
Research on Data Use: A Framework and Analysis
Coburn, C. E., & Turner, E. O. (2011) · Measurement: Interdisciplinary Research and Perspectives · doi:10.1080/15366367.2011.626729 · Cited by 365 · see citing works
Kerr, K. A., Marsh, J. A., Ikemoto, G. S., Darilek, H., & Barney, H. (2006) · American Journal of Education · doi:10.1086/505057 · Cited by 341 · see citing works
Interventions Promoting Educators’ Use of Data: Research Insights and Gaps
Marsh, J. A. (2012) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1177/016146811211401106 · Cited by 328 · see citing works
Data in Practice: Conceptualizing the Data-Based Decision-Making Phenomena
Spillane, J. P. (2012) · American Journal of Education · doi:10.1086/663283 · Cited by 308 · see citing works
Leading Schools in a Data-Rich World
Earl, L. M., & Katz, S. (2002) · Springer eBooks · doi:10.1007/978-94-010-0375-9_34 · Cited by 284 · see citing works
Teachers’ Use of Data: Loose Coupling, Agenda Setting, and Team Norms
Young, V. M. (2006) · American Journal of Education · doi:10.1086/505058 · Cited by 281 · see citing works
The Practice of Data Use: An Introduction
Coburn, C. E., & Turner, E. O. (2012) · American Journal of Education · doi:10.1086/663272 · Cited by 254 · see citing works
Creating a system for data-driven decision-making: applying the principal-agent framework
Wohlstetter, P., Datnow, A., & Park, V. (2008) · School Effectiveness and School Improvement · doi:10.1080/09243450802246376 · Cited by 248 · see citing works
Understanding Data Use Practice among Teachers: The Contribution of Micro-Process Studies
Little, J. W. (2012) · American Journal of Education · doi:10.1086/663271 · Cited by 220 · see citing works
Making Sense of Data-Driven Decision Making in Education
Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006) · Cited by 204 · see citing works
Making Sense of Data-Driven Decision Making in Education: Evidence from Recent RAND Research
(2006) · RAND Corporation eBooks · doi:10.7249/op170 · Cited by 202 · see citing works
Park, V., & Datnow, A. (2009) · School Leadership & Management · doi:10.1080/13632430903162541 · Cited by 199 · cited 6× · see citing works
“Collaboration, trust, and the willingness and capability to address conflict are necessary ingredients for the use of data (Copland, 2003; Datnow et al., 2013; Horn & Little, 2010; Means et al., 2010; Nelson & Slavit, 2007; Park & Datnow, 2009).”
BackgroundInfluentialData-based Decision Making in Education
Schildkamp, K., Lai, M. K., & Earl, L. M. (2012) · Springer eBooks · doi:10.1007/978-94-007-4816-3 · Cited by 195 · see citing works
Self-citation(2007) · Cited by 192 · see citing works
Supovitz, J. A., & Klein, V. (2003) · PsycEXTRA Dataset · doi:10.1037/e382752004-001 · Cited by 192 · see citing works
Marsh, J. A., Sloan McCombs, J., & Martorell, F. (2010) · Educational Policy · doi:10.1177/0895904809341467 · Cited by 181 · see citing works
Teachers' Ability to Use Data to Inform Instruction: Challenges and Supports.
Means, B., Chen, E. E., DeBarger, A. H., & Padilla, C. (2011) · Cited by 178 · see citing works
(2010) · US Department of Education · Cited by 176 · see citing works
Data teams for school improvement
Schildkamp, K., Poortman, C. L., & Handelzalts, A. (2016) · School Effectiveness and School Improvement · doi:10.1080/09243453.2015.1056192 · Cited by 171 · cited 1× · see citing works
“The data team members go back and forward between the steps (see also Figure 1) (Schildkamp, Handelzalts, & Poortman, 2012; Schildkamp & Ehren, 2012, p. 56–57): 1.”
BackgroundSelf-citation(2008) · Cited by 167 · see citing works
Data-based Decision Making: An Overview
Lai, M. K., & Schildkamp, K. (2013) · Data-based Decision Making in Education · doi:10.1007/978-94-007-4816-3_2 · Cited by 167 · see citing works
Self-citationLevin, J. A., & Datnow, A. (2012) · School Effectiveness and School Improvement · doi:10.1080/09243453.2011.599394 · Cited by 159 · see citing works
Affordances and constraints in the context of teacher collaboration for the purpose of data use
Datnow, A., Park, V., & Kennedy‐Lewis, B. (2013) · Journal of Educational Administration · doi:10.1108/09578231311311500 · Cited by 148 · see citing works
Data, Dyads, and Dynamics: Exploring Data Use and Social Networks in Educational Improvement
Daly, A. J. (2012) · Teachers College Record The Voice of Scholarship in Education · doi:10.1177/016146811211401103 · Cited by 142 · see citing works
Using data to support educational improvement
Campbell, C., & Levin, B. (2008) · Educational Assessment Evaluation and Accountability · doi:10.1007/s11092-008-9063-x · Cited by 133 · see citing works
School–Central Office Relationships in Evidence Use: Understanding Evidence Use as a Systems Problem
Honig, M. I., & Venkateswaran, N. (2012) · American Journal of Education · doi:10.1086/663282 · Cited by 133 · see citing works
Organizational considerations in establishing the Data-Informed District
Wayman, J. C., Jimerson, J. B., & Cho, V. (2012) · School Effectiveness and School Improvement · doi:10.1080/09243453.2011.652124 · Cited by 122 · see citing works
Data-Informed Leadership in Education.
Knapp, M. S., Swinnerton, J. A., Copland, M. A., & Monpas-Huber, J. (2006) · Cited by 120 · see citing works
Collaborative inquiry to make data-based decisions in schools
Huffman, D., & Kalnin, J. S. (2003) · Teaching and Teacher Education · doi:10.1016/s0742-051x(03)00054-4 · Cited by 105 · see citing works
Breiter, A., & Light, D. (2006) · Cited by 103 · see citing works
Alternative quality standards in qualitative research?
Poortman, C. L., & Schildkamp, K. (2012) · Quality & Quantity · doi:10.1007/s11135-011-9555-5 · Cited by 100 · see citing works
Self-citationData Use for School Improvement: School Practices and Research Perspectives
Wayman, J. C., & Stringfield, S. (2006) · American Journal of Education · doi:10.1086/505055 · Cited by 98 · see citing works
Identifying and Monitoring Students' Learning Needs With Technology
Chen, E., Heritage, M., & Lee, J. (2005) · Journal of Education for Students Placed at Risk (JESPAR) · doi:10.1207/s15327671espr1003_6 · Cited by 93 · see citing works
Professional development to promote schoolwide inquiry
King, M. B. (2002) · Teaching and Teacher Education · doi:10.1016/s0742-051x(01)00067-1 · Cited by 89 · see citing works
The development and testing of a school improvement model
Leithwood, K. A., Jantzi, D., & McElheron-Hopkins, C. (2006) · School Effectiveness and School Improvement · doi:10.1080/09243450600743533 · Cited by 84 · see citing works
School performance feedback systems in the USA and in The Netherlands: a comparison
Schildkamp, K., & Teddlie, C. (2008) · Educational Research and Evaluation · doi:10.1080/13803610802048874 · Cited by 76 · see citing works
Self-citationSustaining continued acceleration in reading comprehension achievement following an intervention
Lai, M. K., McNaughton, S., Timperley, H. S., & Hsiao, S. (2009) · Educational Assessment Evaluation and Accountability · doi:10.1007/s11092-009-9071-5 · Cited by 64 · see citing works
Tough Choices in Designing a Formative Assessment System
Sharkey, N. S., & Murnane, R. J. (2006) · American Journal of Education · doi:10.1086/505060 · Cited by 60 · see citing works
Nelson, T. H., & Slavit, D. (2007) · Journal of In-Service Education · doi:10.1080/13674580601157620 · Cited by 59 · see citing works
Conclusions and a Data Use Framework
Schildkamp, K., & Lai, M. K. (2012) · Springer eBooks · doi:10.1007/978-94-007-4816-3_10 · Cited by 57 · see citing works
Self-citationAttitudes towards school self-evaluation
Vanhoof, J., Petegem, P. V., & Mæyer, S. D. (2009) · Studies In Educational Evaluation · doi:10.1016/j.stueduc.2009.01.004 · Cited by 56 · see citing works
From “Intuition”- to “Data”-based Decision Making in Dutch Secondary Schools?
Schildkamp, K., & Ehren, M. (2012) · Springer eBooks · doi:10.1007/978-94-007-4816-3_4 · Cited by 55 · see citing works
Self-citationApprenticeship: from learning theory to practice
Poortman, C. L., Illeris, K., & Nieuwenhuis, L. F. M. (2011) · Journal of Vocational Education and Training · doi:10.1080/13636820.2011.560392 · Cited by 50 · see citing works
Self-citationchapter 4 Understanding the Promise and Dynamics of Data‐Informed Leadership
Knapp, M. S., Copland, M. A., & Swinnerton, J. A. (2007) · Yearbook of the National Society for the Study of Education · doi:10.1111/j.1744-7984.2007.00098.x · Cited by 47 · see citing works
Understanding the Promise and Dynamics of Data-Informed Leadership
Knapp, M. S., Copland, M. A., & Swinnerton, J. A. (2007) · Teachers College Record The Voice of Scholarship in Education · Cited by 46 · see citing works
Data-driven School Improvement
(2007) · SAGE Publications Ltd eBooks · doi:10.4135/9781446214596.n18 · Cited by 44 · see citing works
Student group differences in examination results and utilization for policy and school development
Schildkamp, K., Rekers-Mombarg, L. T. M., & Harms, T. J. (2012) · School Effectiveness and School Improvement · doi:10.1080/09243453.2011.652123 · Cited by 35 · see citing works
Self-citationSchaffer, E., Reynolds, D., & Stringfield, S. (2012) · Journal of Education for Students Placed at Risk (JESPAR) · doi:10.1080/10824669.2012.637188 · Cited by 31 · see citing works
(2002) · Cited by 26 · see citing works
(2012) · ProQuest LLC eBooks · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Mandinach, E. B., & Gummer, E. S. (2016) · Teaching and Teacher Education · doi:10.1016/j.tate.2016.07.011 · Cited by 363 · see citing works
Data-based decision-making for school improvement: Research insights and gaps
Schildkamp, K. (2019) · Educational Research · doi:10.1080/00131881.2019.1625716 · Cited by 347 · see citing works
Self-citationFormative assessment: A systematic review of critical teacher prerequisites for classroom practice
Schildkamp, K., van der Kleij, F. M., Heitink, M. C., Kippers, W. B., & Veldkamp, B. P. (2020) · International Journal of Educational Research · doi:10.1016/j.ijer.2020.101602 · Cited by 330 · see citing works
Self-citationDatnow, A., & Hubbard, L. (2016) · Journal of Educational Change · doi:10.1007/s10833-015-9264-2 · Cited by 287 · see citing works
Misconceptions about data-based decision making in education: An exploration of the literature
Mandinach, E. B., & Schildkamp, K. (2021) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2020.100842 · Cited by 209 · see citing works
Self-citationData teams for school improvement
Schildkamp, K., Poortman, C. L., & Handelzalts, A. (2016) · School Effectiveness and School Improvement · doi:10.1080/09243453.2015.1056192 · Cited by 171 · see citing works
Self-citationVan der Kleij, F. M., Vermeulen, J. A., Schildkamp, K., & Eggen, T. J. H. M. (2015) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969594x.2014.999024 · Cited by 135 · see citing works
Self-citationSolving student achievement problems with a data use intervention for teachers
Poortman, C. L., & Schildkamp, K. (2016) · Teaching and Teacher Education · doi:10.1016/j.tate.2016.06.010 · Cited by 95 · see citing works
Self-citationThe effects of a data use intervention on educators’ satisfaction and data literacy
Ebbeler, J., Poortman, C. L., Schildkamp, K., & Pieters, J. M. (2017) · Educational Assessment, Evaluation and Accountability · doi:10.1007/s11092-016-9251-z · Cited by 77 · see citing works
Self-citationData literacy training and use for educational professionals
Henderson, J., & Corry, M. (2021) · Journal of Research in Innovative Teaching & Learning · doi:10.1108/jrit-11-2019-0074 · Cited by 73 · see citing works
Data-based decision making for instructional improvement in primary education
Gelderblom, G., Schildkamp, K., Pieters, J., & Ehren, M. (2016) · International Journal of Educational Research · doi:10.1016/j.ijer.2016.07.004 · Cited by 74 · see citing works
Self-citationEffects of a data use intervention on educators’ use of knowledge and skills
Ebbeler, J., Poortman, C. L., Schildkamp, K., & Pieters, J. M. (2016) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2015.11.002 · Cited by 53 · see citing works
Self-citationPowell, S. R., Lembke, E. S., Ketterlin-Geller, L. R., Petscher, Y., Hwang, J., Bos, S. E., Cox, T., Hirt, S., Mason, E. N., Pruitt-Britton, T., et al. (2021) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2020.100897 · Cited by 37 · see citing works
Datnow, A., Lockton, M., & Weddle, H. (2021) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2020.100869 · Cited by 31 · see citing works
Exploring the factors influencing teachers’ instructional data use with electronic data systems
Luo, J., Wang, M., & Yu, S. (2022) · Computers & Education · doi:10.1016/j.compedu.2022.104631 · Cited by 32 · see citing works
Albiladi, W., Lasater, K., & Bengtson, E. (2020) · Journal of School Administration Research and Development · doi:10.32674/jsard.v5i2.2240 · Cited by 12 · see citing works
The Nature of Teacher Learning in Collaborative Data Teams
Michaud, R. (2016) · The Qualitative Report · doi:10.46743/2160-3715/2016.2316 · Cited by 2 · see citing works
- Authors
- Schildkamp, K., & Poortman, C.
- Affiliations
- University of Twente (NL)
- Venue
- Teachers College Record: The Voice of Scholarship in Education
- Publisher
- SAGE Publications
- Published
- 2015-04
- Volume
- 117
- Issue
- 4
- Pages
- 1-42
- ISSN
- 0040-0475 (ISSN-L)
- 0161-4681
- 1467-9620
- OpenAlex
- W272924859
- Cited by
- 185
- References
- 66
- Citations by year
- 2026: 11 · 2025: 16 · 2024: 16 · 2023: 15 · 2022: 23 · 2021: 16 · 2020: 23 · 2019: 17 · 2018: 7 · 2017: 15 · 2016: 18 · 2015: 7 · 2014: 1
- FWCI
- 30.78
- Type
- journal article
- Language
- en
- Open access
- green
- research.utwente.nl
- Locations
- Teachers College Record The Voice of Scholarship in Education (publishedVersion)
- University of Twente Research Information (publishedVersion, other-oa)
- University of Twente Research Information (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Educational Assessment and Improvement
- Social Sciences › Education › Teacher Education and Leadership Studies
- Social Sciences › Education › Evaluation of Teaching Practices
- Keywords
- data teams, data-driven decision making, data literacy, pedagogical content knowledge, school leadership, upper secondary school, data characteristics, school organization, university-school partnership
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
