Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors

“Black and Wiliam (1998b) found, for example, that assessment data are usually not shared between teachers within schools, nor are they critically reviewed in relation to what they actually assess.”

“School staff can also abuse data or use data for educational triage practices (Gilborn & Youdell, 2000 cited in Booher-Jennings, 2005) or the focus on so-called bubble kids (Booher-Jennings, 2005).”

“…conducted in this area and reflects an international perspective (Datnow, Park, & Kennedy-Lewis, 2012; Jimerson, 2013; Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006; Schildkamp et al., 2013; Schildkamp & Kuiper, 2010; Schildkamp & Teddlie, 2008; Vanhoof, Verhaeghe, Van Petegem, & Valcke, 2012).”

“For example, this may include difficulties with analysis and/or translating the data into an action plan (e.g. Brown, Schildkamp, and Hubers 2017; Schildkamp and Kuiper 2010; Schildkamp and Poortman 2015; Schildkamp, Poortman, and Handelzalts 2016).”

“Scholarship suggests that these different paths of attribution have implications for instruction and learning (Jussim & Harber, 2005; Schildkamp & Kuiper, 2010).”

“…types of analysis, and even more often they have difficulties with connecting the data to their own instruction in the classroom and translating the data into an action plan (e.g., Brown, Schildkamp, & Hubers, 2017; Schildkamp & Kuiper, 2010; Schildkamp & Poortman, 2015; Schildkamp et al., 2016).”

“…for Improving School and Student Performance, 2011; Geijsel, Sleegers, Stoel, & Krüger, 2009; Kelly, Downey, & Rietdijk, 2010; Schildkamp, 2007; Schildkamp & Kuiper, 2010; Schildkamp & Teddlie, 2008; Vanhoof, Verhaeghe, Verhaeghe, Valcke, & Van Petegem; 2011; Verhaeghe, Vanhoof, Valcke, & Van…”

“…the literature:  Access to high quality data (Coburn & Turner, 2011; Kerr et al., 2006; Mandinach & Honey, 2008; Marsh et al., 2006; Means, et al., 2010; Schildkamp & Kuiper, 2010; Sharkey & Murnane, 2006; Wayman et al., 2006; Wayman & Stringfield 2006; Wayman, et al., 2007; Wayman et al., 2012).”

“…& Poortman, 2015, p. 5) is based on the analyses of several data use models and frameworks (Coburn & Turner, 2011; Ikemoto & Marsh, 2007; Lai & Schildkamp, 2013; Mandinach, Honey, Light, & Brunner, 2008; Marsh, 2012; Schildkamp et al., 2012; Schildkamp & Kuiper, 2010; Schildkamp & Poortman, 2015).”

“The data for the Dutch case studies were collected in a previous study (Schildkamp & Kuiper, 2010).”

“Collaborating with colleagues in generating knowledge relevant to student learning has been identified in a number of studies related to data use as a capacitybuilding approach (Datnow et al., 2007; Lachat & Smith, 2005; Kerr et al., 2006; Schildkamp & Kuiper, 2010; Wayman & Stringfield, 2006).”

“Data can also be used for formative evaluation by school leaders and teachers for policy development and school improvement planning, teacher development and monitoring the implementation of the school ’ s goals (Schildkamp et al., 2013; Schildkamp & Kuiper, 2010).”

“Complicating matters further, data use is not just one practice: different forms of data may require different practices (Schildkamp & Kuiper, 2010).”

“DBDM refers to goal setting as well as the collection and use of quantitative and/or qualitative data, after which actions for improvement are designed and implemented (e.g. Schildkamp and Kuiper 2010 ).”

“While the positive effects of collaboration are likewise well established in the data-use literature (e.g., Kerr et al., 2006; Lachat & Smith, 2005; Schildkamp & Kuiper, 2010; Wayman & Stringfield, 2006), the link between collaboration and learning how to use data is just beginning to emerge.”

“Educators have been shown collaborating around data in a variety of ways, such as grade-level teams, subject-level teams, or professional learning communities (Kerr et al., 2006; Lachat & Smith, 2005; Schildkamp & Kuiper, 2010; Wayman & Stringfield, 2006).”

“They can then use the outcome of such an evaluation to determine how they can further improve the quality of their lessons (Schildkamp & Kuiper, 2010).”

“This is presumed to lead to improvements in educational processes and increased student achievement (e.g. Earl & Katz, 2006; Schildkamp & Kuiper, 2010; Vanhoof & Schildkamp, 2014; Young, 2006).”

“…in professional development linking data to instructional practice (Cosner, 2011; Datnow et al., 2007; Kerr et al.,2006; Saunders et al., 2009; Schildkamp & Kuiper, 2010; Supovitz, 2002; Wayman & Cho, 2007) or address data literacy needs through preservice preparation programs (Mandinach &…”

“…practice is often social in nature, (e.g., Datnow, Park, & Wohlstetter, 2007; Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006; Lachat & Smith, 2005; Schildkamp & Kuiper, 2010; Wayman & Stringfield, 2006) and can result in better instructional decisionmaking and foster deeper use of data (Lachat &…”

“Despite the disengagement of teachers from computerised pupil progress monitoring noted by Schildkamp and Kuiper (2010), given its use in monitoring teacher performance, and the manipulation of data in that event (Ball 2003), SEND-related statutory guidance has ensured that some teachers will spend…”

“Although these summative results may be important in assessing a certain type of student achievement, research suggests these assessments may not be as useful in making more timely day to day instructional decisions (Earl & Katz, 2007; Schildkamp & Kuiper, 2010).”

“Based on Marsh (2012, p. 4), and incorporating relevant characteristics of other data use models and frameworks (Coburn and Turner 2011; Ikemoto and Marsh 2007; Schildkamp and Lai (2013); Mandinach et al. 2008; Schildkamp and Kuiper 2010; Supovitz 2010), a data team theory of action (Fig.”

“…towards data use) (see, e.g., Ikemoto & Marsh, 2007; Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006; Levin & Datnow, this issue; Schildkamp et al., this issue; Schildkamp & Kuiper, 2010; Schildkamp & Teddlie, 2008, Schildkamp & Visscher, 2009; Visscher, 2002; Wayman, 2005; Wohlstetter et al., 2008).”

“…cite difficulties due to a lack of knowledge of how to effectively use data and a scarcity of quality supports for increasing this knowledge (Louis et al., 2010; Ikemoto & Marsh, 2007; Mandinach & Gummer, 2013; Mandinach & Jackson, 2012; Schildkamp & Kuiper, 2010; Wayman, Cho, & Johnston, 2007).”

“It can consist of assessment data, inspection data, observations, background information on students, and so forth (Ikemoto & Marsh, 2007; Schildkamp & Kuiper, 2010).”

“Data have to be identified, collected, analyzed, and interpreted before actions can be taken (Ikemoto & Marsh, 2007; Schildkamp & Kuiper, 2010; Van der Kleij, Vermeulen, Schildkamp, & Eggen, 2015).”

“According to Schildkamp and Kuiper (2010), school leaders and school policies can contribute to efficient and successful data handling by teachers.”

“The act of interpreting data is what leads to information that “has meaning, relevance, and purpose” (Schildkamp & Kuiper, 2010, p. 482).”

“teacher (Schildkamp and Kuiper 2010; Ledoux et al. 2009).”

“These are evidence derived from practitioner inquiry, such as action research (e.g. Stoll, Earl, Anderson, & Schildkamp, 2014); and evidence derived from routinely collected school- or system-level data, for example pupil assessment data (e.g. see Schildkamp & Kuiper, 2010).”

“Instead, they frequently base their practice on tradition, common knowledge, and experience (Cain, 2016; Hargreaves, 2000; Hetmanek et al., 2015; Kutash, Duchnowski & Lynn, 2009; Schildkamp & Kuiper, 2010; van Schaik et al., 2018).”

“Data-based decision-making in education has become increasingly important in a number of countries (Datnow, Park, and Kennedy-Lewis 2013; Schildkamp and Kuiper 2010).”

“(Schildkamp & Kuiper, 2010) The purpose of the inquiry was to examine the nature and benefits of systematic inquiry in schools and to understand the forces that serve to enhance or impede schools’ propensity to embrace evaluative inquiry as a support for decision making.”

“DDDM critics emphasize the need to understand a wider context beyond high-stakes test scores (Schildkamp & Kuiper, 2010).”

“Scholars have focused on how teachers use data to improve their instruction and students’ academic achievement (Jennings, 2012; Liou et al., 2014; Little, 2012; Schildkamp & Kuiper, 2010; Wohlstetter, Datnow, & Park, 2008).”

“Much of the research to date has dealt with specific areas of schools, such as teachers (Lachat and Smith 2005; Schildkamp and Kuiper 2010), principals (Supovitz and Klein 2003; Wayman, Cho, and Johnston 2007) and the central office (Honig and Coburn 2008; Moll 2009).”

“The inspectorate expects schools to continuously use data to improve their education (Schildkamp & Kuiper, 2010).”

“…do something with it”: teacher agency in efforts to bridge the gap between data use and instructional improvement Consistent with prior research, most teachers did not find accountability-driven forms of assessment to be useful in planning instruction (Kerr et al., 2006; Schildkamp & Kuiper, 2010).”

“E-mail: sacosner@uic.edu 26 Schildkamp & Kuiper, 2010; Steele & Boudett, 2008; Timperley, Annan, & Robinson, 2009).”

“After the training, teachers can analyse data available at their school to implement and design some improvement measures (Borko, 2004; Schildkamp and Kuiper, 2010).”

“…these purposes frequently make mistakes in the interpretation of certain outputs (Van der Kleij & Eggen, 2013) and are often unaware of the possibilities for more sophisticated analyses (Ledoux, Blok, & Boogaard, 2009; Meijer & Ledoux, 2011; Schildkamp & Kuiper, 2010; Van der Kleij & Eggen, 2013).”

“…(e.g., Hat-tie & Timperley, 2007), as “data can help teachers to monitor their constantly changing environment, their functioning and to what extent curriculum aims are met, and react timely and in an evidence-based manner when problems need to be solved” (Schildkamp & Kuiper, 2010, p. 482).”

“This could enhance student learning if teachers reflect (more) on the impact of the instruction they have given, and attempt to provide tailor-made instruction based on their knowledge of how students perform and which learning problems they face (Schildkamp & Kuiper, 2010).”

“…2008; Ikemoto & Marsh, 2005; Kelly & Downey, 2010; Kelly, Downey, & Rietdijk, 2010; Kerr et al., 2006; Schildkamp, 2007; Schildkamp et al., 2011; Schildkamp & Kuiper, 2010; Schildkamp, Lai, & Earl, in press; Schildkamp & Teddlie, 2008; Schildkamp & Visscher, 2009; Schildkamp, Visscher, &…”

“Teams should function more openly and be willing to reflect critically on their functioning supported by (self‐evaluation) data (Schildkamp & Kuiper, 2010 ).”

“…‘data use’, ‘data-based decision-making’ (Lai & Schildkamp, 2013; Mandinach & Schildkamp, 2021; Schildkamp et al., 2014), ‘data-driven decision-making’ (Mandin-ach, 2012; Sampson et al., 2022; Schildkamp & Kuiper, 2010), or ‘data-informed decision-making’ (Shen & Cooley, 2008) are most common.”

“As a result, teachers may change their behavior, such as by trying out different instructional strategies, which can improve their own performance (Schildkamp and Kuiper, 2010) and lead to improved student achievement (Campbell and Fullan, 2006; Carlson et al., 2011; Lai et al., 2009).”

“…a positive attitude towards data use at the start may also enable the work of data teams (Coburn & Turner, 2011; Earl & Katz, 2006; Levin & Datnow, 2012; Mandinach & Honey, 2008; Marsh, 2012; Marsh et al., 2006; Schildkamp & Kuiper, 2010; Spillane, 2012; Vanhoof, Van Petegem, & De Maeyer, 2009).”

“However, it is precisely this attitude towards data use that is a key factor in explaining differences in data use in schools (Ikemoto and Marsh 2007; Schildkamp and Kuiper 2010).”

“school performance, and, implementing (e.g. genuine improvement actions) and evaluating these innovations ” (Schildkamp & Kuiper, 2010, p. 482).”

“Using data then implies interpreting it and so transforming it into information (cf. Schildkamp and Kuiper, 2010, p. 482).”

“2004), teacher capacity to adjust practice based on data (Lai and McNaughton, this book; Schildkamp and Kuiper 2010; Wayman and Stringfield 2006), and district supports to enable these components (Datnow et al.”

“Earlier research has shown that psychological factors can contribute to explaining differences in the use of data between school leaders and between teachers (Ikemoto and Marsh 2007; Schildkamp and Kuiper 2010; Vanhoof et al. 2014).”