Data teams for school improvement

May combine two claims.

Team B remained far behind; most meetings focused on defining problem, criteria, personal experiences and opinions about solutions.

“Team B remained far behind: Most of their meetings were only focused on defining the problem and setting related criteria and personal experiences and opinions about possible solutions.”

The quote only partly states this claim.

When Team B moved to hypotheses and data collection, many proposed data were not available.

“When finally moving on to hypotheses and collecting data, many of the proposed data were not available.”

May combine two claims.

Only Team C and Team W actually investigated hypotheses related to own functioning and took action at this level.

“Only Team C and Team W, however, actually investigated hypotheses related to their own functioning, and consequently took action at this level.”

May combine two claims.

Team G had only six meetings within pilot study and did not reach level of turning knowledge into action and outcomes.

“They only had six meetings within the period of the pilot study, within which they did not reach the level of turning knowledge into action and outcomes.”

May combine two claims.

Three of four teams went through several feedback loops and developed new knowledge based on data.

“Three of the four teams went through several feedback loops and were able to develop new knowledge based on data.”

May combine two claims.

Data do not always directly lead to action, but data challenged assumptions of data team members.

“Moreover, the data do not always directly lead to action, but the data have challenged the assumptions of the data team members.”

May combine two claims.

Interrater agreement was satisfactory, with Cohen's Kappa of 0.76.

“We calculated the interrater agreement and found a satisfying Cohen's Kappa of 0.76.”

May combine two claims.

The study's findings show that data use is not linear and that teams move through different feedback loops to reach deeper levels of inquiry.

“The findings show that data use is not a linear process, and that teams go through different feedback loops to reach higher levels of depth of inquiry.”

May combine two claims. The quote only partly states this claim.

Team C's evaluation after a couple months showed students advanced considerably from pretest to first posttest, and results did not significantly decline from first to second posttest; later the paper states results did not significantly decline in percentages and fractions anymore.

“The evaluation after a couple of months showed the outcome that students advanced considerably from the pretest to the first posttest, and subsequently student results did not significantly decline from the first to the second posttest.”

• Also stated in: § M1 M2 · plain

  “Team C implemented several measures to improve mathematic achievement, and the outcome was that student results did not significantly decline in percentages and fractions anymore.”

May combine two claims.

In Team G's first two meetings, strong feeling that lower level intake in middle-level stream was problematic guided hypothesis formulation, and hypotheses were not supported by data.

“In the first two meetings, the strong feeling that the lower level intake in the middle-level stream was problematic guided the team in formulating hypotheses.”

• Also stated in: § Team G: depth of inquiry · plain

  “Although these hypotheses were not supported by the data, several team members returned to this feeling in later meetings.”

May combine two claims. The quote only partly states this claim.

In Team C's first two meetings, members talked about problem based on own experiences, not data; later meetings used more data and reached higher depth.

“In the first two meetings, team members talked about the problem based on their own experiences, and not based on data.”

• Also stated in: § Team C: depth of inquiry · plain

  “From Meetings 6 to 11, more data were collected and used as a basis for drawing conclusions about the problem, hypotheses, and solutions.”

May combine two claims.

By Meeting 4, Team W understood that the problem had multiple causes and that they needed to examine internal causes such as curriculum and their own functioning.

“The teachers reached two important understandings by the time of Meeting 4: (a) the problem was caused by more than one issue, and (b) they needed to start looking at internal causes, such as problems with their curriculum and their own functioning.”

The quote only partly states this claim.

More advanced analyses by external facilitator showed Team C's scores dropped considerably.

“The more advanced analyses, carried out by the external facilitator, showed that the scores dropped considerably.”

Probably not a substantial claim: may not pass S3 (load-bearing).

With help of facilitator, it became clear there were too few cases to draw meaningful conclusions for Team G.

“With help of the facilitator, it became clear that there were too few cases to draw meaningful conclusions.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Team B mentioned external causes and sometimes its own functioning from the start.

“The teachers further mentioned many external possible causes for the problem, such as the difficulty of mathematics or the (starting) level of students.”

• Also stated in: § Team B: attribution · plain

  “However, they sometimes also brought up their own functioning from the start.”

May combine two claims.

Team B's depth of inquiry was confined to lowest levels and typically did not exceed average.

“This means that depth of inquiry was confined to the lowest levels in the meetings, and typically did not exceed the”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.

This result surprised Team G; their gut feeling was wrong and it was even the other way around.

“This came as a surprise to the team:”

May combine two claims.

Team W's data showed that hypotheses about transfer-rule strictness and motivation were incorrect.

“Other hypotheses concerned strictness of transfer rules and motivation.Data showed that these hypotheses were incorrect.”

May combine two claims.

Team W's motivation survey showed that both failing and passing students needed help with planning and more teacher feedback on planning.

“However, the motivation survey results showed that students (both in the failing and passing group) needed help in planning and needed more feedback from teachers on their planning activities.”

May combine two claims.

In Meeting 5, Team W compared failing and passing students, related answers to questions, discussed its own functioning, and developed new knowledge from data at high depth.

“In Meeting 5, the team compared the results of failing and passing students and tried to relate the answers to different questions; also discussing their own functioning in relation to the students' opinions about this.The team was developing new knowledge based on the collected data (high depth).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.

Team C's data related to suspected problem were not readily available, and teachers had to construct and coordinate tests.

“One of the challenges was that the data related to the suspected problem were not readily available, and the teachers had to first construct and coordinate the tests to find out more about the problem.”

May combine two claims.

In its first meetings, Team W mainly considered hypotheses at the school level, though some hypotheses about student level and their own functioning were also considered.

“Although some hypotheses about their own functioning and at the level of students were considered, the team mainly focused on hypotheses at the school level in the first meetings.”

May combine two claims.

Team B acknowledged the problem only as late as the fifth meeting, and not convincingly or explicitly.

“After extensive discussion, the problem was finally, although not convincingly or explicitly, acknowledged as late as the fifth meeting.”

May combine two claims.

Teams generally started discussing problem from own experience, investigating more external hypotheses.

“With regard to purpose, the teams generally started discussing the problem from their own experience, investigating the more external hypotheses.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In Meeting 8, Team W found absenteeism explained a substantial part of the problem and concluded absenteeism measures should improve.

“In Meeting 8, the team started with discussing the absenteeism results.It turned out that this explained a substantial part of the problem, and that the school's absenteeism measures should be improved.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Team W found that teachers did not regularly check homework.

“Additionally, it seemed that teachers did not check homework regularly.”

May combine two claims. The quote only partly states this claim.

By final meetings, Team C teachers had structurally implemented repetition in lessons for fractions and percentages and used online programs to differentiate instruction.

“By the time of the final meetings, the teachers had structurally implemented the repetition measure in their lessons, both for fractions and percentages, and were using the online programs to differentiate instruction.”

May combine two claims.

Team W's depth of inquiry was lowest in Meetings 1 and 2, focused on personal experiences and anecdotes.

“Depth of inquiry was nearly all the time confined to the lowest levels in Meetings 1 and 2: The discussion focused mostly on own personal experiences and anecdotes.”

May combine two claims.

In Team W, members generally contributed actively, asked clarifying questions, and complemented one another.

“In general, the different members of the team actively contributed to the discussions, asked each other clarifying questions, and complemented each other.”

May combine two claims.

Teams W and C managed to successfully complete Steps 1 to 8 within pilot, create new knowledge, and base related decisions on data.

“Teams W and C managed to successfully complete Steps 1 to 8 within the pilot study period, create new knowledge, and base related decisions on data.”

May combine two claims.

Some data teams reached the action and outcomes phase and were able to use data to make improvements, while others did not.

“The findings show that our theory of action seems to be applicable in practice, but also that teachers and school leaders go through different feedback loops, and that some are able to reach the action and outcomes phase, whereas others are not.”

• Also stated in: § Implications for practice and recommendations for further research · plain

  “Some teams were able to engage and use data to make improvements, whereas others were not.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Data use for educational decision making has become more prevalent than ever.

“The use of data for educational decision making has never been more prevalent”

• cites Datnow, Park, & Kennedy-Lewis, 2013 sourceNot checked — no full text of the source
• cites Marsh & Farrell, 2015 sourceNot checked — no full text of the source
• cites Schildkamp & Kuiper, 2010 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

In 2010, only 20% of Dutch secondary schools applied data use according to the Dutch Inspectorate.

“In 2010, only 20% of the Dutch secondary schools applied data use according to the Dutch Inspectorate”

• cites Inspectie van het Onderwijs, 2011Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

In more successful teams, teachers examine underlying assumptions, question their practice, revise their ideas about teaching and learning, and change their practice.

“In these teams, teachers explore underlying assumptions, question their own practice, revise their conceptions of teaching and learning, and adjust their practice accordingly.”

May combine two claims.

Looking for external causes does not solve the problem because teachers often cannot change those causes.

“Moreover, it does not lead to solving the problem as often there is nothing that the teachers can do about this external cause.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The Dutch Inspectorate increasingly holds schools accountable for using data to improve education quality.

“the Dutch Inspectorate increasingly holds schools accountable for making use of data to improve the quality of their education”

• cites Verbeek & Odenthal, 2014Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

The data use theory of action was built from analyses of several data use models and frameworks.

“Our data use theory of action (Schildkamp & Poortman, 2015, p. 5) is based on the analyses of several data use models and frameworks”

• cites Coburn & Turner, 2011 sourceNot checked — no full text of the source
• cites Ikemoto & Marsh, 2007 sourceNot checked — no full text of the source
• cites Lai & Schildkamp, 2013 sourceNot checked — no full text of the source

May combine two claims.

Data teams focus on professional development in data use and school improvement, measured ultimately by student achievement outcomes, rather than on teachers building their own practical theories.

“The focus of data teams, however, is on professional development specifically in data use and school improvement, ultimately measured in student achievement outcomes; rather than on teachers building their own practical theories of teaching”

• cites Whitehead & McNiff, 2006 sourceNot checked — no full text of the source

May combine two claims.

Looking for causes outside the school can help teams learn to use data, but such assumptions are often wrong.

“Investigating causes outside the school contributes to the learning process (learning how to use data), but often these assumptions are incorrect”

• cites Ingram et al., 2004 sourceNot checked — no full text of the source

May combine two claims.

Teachers often do not use data as well as they could, or do not use it at all.

“teachers often do not use data to its best effect, if at all”

• cites Schildkamp & Kuiper, 2010 sourceNot checked — no full text of the source
• cites Schildkamp & Teddlie, 2008 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable).

A qualitative descriptive approach is best for describing how data teams function in their context.

“A qualitative, descriptive approach is most useful to describe the process of the functioning of data teams in its context”

• cites Krathwohl, 1998Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Data-based decision making is complex and non-linear and involves multiple feedback loops.

“The framework acknowledges that data-based decision making is a complex and non-linear process, involving several feedback loops.”

May combine two claims.

Collaboration enables teachers to learn from one another about data use and to exchange ideas and strategies.

“Collaboration helps teachers to learn from each other how to use data, and allows for a fertile exchange of ideas and strategies”

• cites Nelson & Slavit, 2007 sourceNot checked — no full text of the source
• cites Wohlstetter, Datnow, & Park, 2008 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Recording whole conversations gives more depth and specificity than using observation protocols or field notes alone.

“Capturing the entire conversations in the meetings provides much more depth and specificity than we would achieve using observation protocols or field notes alone”

• cites Little, 2012 sourceNot checked — no full text of the source

May combine two claims.

The inquiry element of professional learning communities is crucial because it can improve instruction and thereby raise student achievement.

“According to Ball and Cohen (1999), the inquiry component of PLCs is crucial as this can improve instruction, which in turn can lead to increased student achievement.”

• cites Ball & Cohen, 1999Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Action research is often organized around a cycle of inquiry and can be conducted collaboratively in a professional learning community.

“Action research is often organized around a certain cycle of inquiry (Johnson, 2003) and can also be conducted collaboratively in a PLC (Capobianco & Feldman, 2010).”

• cites Johnson, 2003Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Capobianco & Feldman, 2010 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

How people attribute the cause of a problem has behavioral consequences for improvement actions based on data.

“How individuals attribute causes of a problem has behavioral consequences (Weiner, 2010) in terms of possible improvement actions based on data.”

• cites Weiner, 2010 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The Ministry of Education aims for at least 90% of primary and secondary schools to apply data use by 2018.

“The policy aim of the Ministry of Education is that by 2018, at least 90% of the primary and secondary schools apply data use.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Dutch schools can use several data sources to improve education, including intake, transfer, school leaver, inspection, self-evaluation, assessment, examination, and questionnaire data.

“Dutch schools can use several data sources to improve their education, such as data on intake, transfer, and school leavers; school inspection and selfevaluation data; assessment and examination results; and student and parent questionnaire data”

• cites Schildkamp & Kuiper, 2010 sourceNot checked — no full text of the source

May combine two claims.

Data teams either return to new hypotheses when a hypothesis is wrong or act on the data when a hypothesis is right.

“Data teams can take two possible actions: The hypothesis is incorrect and the action is to go back to formulating new hypotheses (a feedback loop is created), or the hypothesis is correct and the data team takes action based on the data.”

• cites Marsh, Pane, & Hamilton, 2006Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Marsh, 2012 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

In the data use theory of action, data and people interact within a context to produce decisions about what action to take.

“the interaction between data and people, in a certain context, results in decisions regarding what action to take.”

May combine two claims.

Data teams check data validity and reliability; if data are not valid or reliable, they collect more data and create a feedback loop.

“The team needs to filter the data (e.g., are the data valid and reliable; if not, additional data need to be collected and a feedback loop is created)”

Probably not a substantial claim: may not pass S3 (load-bearing).

Researchers who did not facilitate the data teams conducted the analyses to support objectivity.

“For objectivity reasons, the researchers who did not facilitate the data team conducted the analyses.”

Classified as probably not a substantial claim. May combine two claims.

This study gave insight into what occurs when teachers work with data in data teams, opening up the collaborative inquiry process in professional learning communities.

“This study provided more insight into what happens when teachers engage with data in data teams (opening the “black box” of the collaborative inquiry in PLCs; Butler & Schnellert, 2012).”

• cites Butler & Schnellert, 2012 sourceNot checked — no full text of the source

May combine two claims.

The paper cannot conclude improvements were caused by data teams alone, but it is likely data teams contributed.

“Although we cannot conclude that these improvements were caused by the work of the data teams alone, it is likely that the work of the data teams contributed to the improvements.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Team W's school registration data were considered sufficiently reliable and valid.

“The team collected data from their own school registration system, which were considered sufficiently reliable and valid.”

The quote only partly states this claim.

Team C's wrong hypotheses about primary schools and secondary advice led to new knowledge about what did not cause their problem (conceptual data use).

“However, this led to new knowledge with regard to what did not cause their specific problem (conceptual data use).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

All data team meetings and interviews were audio-recorded, which allowed thorough analysis.

“All data team meetings and interviews were audiorecorded, allowing for thorough analysis of the data.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The instruments were based on the theoretical framework and existing instruments.

“Also, the instruments were based on our theoretical framework and existing instruments.”

May combine two claims.

Team W and C actions led to desired outcomes: they solved their educational problem and increased student achievement.

“Moreover, these actions also led to the desired outcomes: These teams were able to solve their educational problem and increase student achievement”

Probably not a substantial claim: may not pass S3 (load-bearing).

As stated by Marsh and Farrell (2015), data team procedures seem promising capacity building interventions.

“As stated by Marsh and Farrell (2015), data team procedures seem to be promising capacity building interventions.”

• cites Marsh and Farrell (2015) sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Two researchers coded about 10% of the same observation transcripts.

“Finally, two researchers coded approximately 10% of the same observation transcripts.”

May combine two claims.

External validity was strengthened by providing case-specific and cross-case thick descriptions and by describing congruence with the theoretical framework.

“We enhanced external validity in terms of analytical generalization by providing casespecific and cross-case thick descriptions (also including citations of respondents), and describing the congruence with the theoretical framework”

• cites Poortman & Schildkamp, 2012 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The coding scheme was refined after an initial sample was coded, through researcher discussions about the scheme's applicability and the meaning of the data.

“After initially coding a sample of the data, the coding scheme was refined, based on discussions between the researchers about the applicability of the scheme and the meaning of the data.”

Classified as probably not a substantial claim.

Organizational context, the characteristics of available data and data systems, macrolevel policy influence, and microlevel personal characteristics of data users all interact in data use.

“We should therefore take into account the organizational context in which data use is taking place, but also the characteristics of the data and data systems available, as well as the influence of policy on data use at the macrolevel, and personal characteristics of the individual data users at the microlevel, as these all interact (Coburn & Turner, 2011, 2012;Marsh, 2012).”

• cites Coburn & Turner, 2011, 2012Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Marsh, 2012 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Contextual factors may explain differences in the practices of different teams.

“As stated by Horn and Little (2010), contextual factors may account for the differences in the practices of different teams.”

• cites Horn and Little (2010) sourceNot checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

Long-term school improvement requires focus, continuous planning, and inquiry into the causes of problems using data.

“Duke (2015) found that long-term school improvement requires focus, continuous planning, and inquiry into the causes of problems (e.g., low performance) using data.”

• cites Duke (2015) sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

These contextual factors may include school organizational characteristics, such as the school leader's role, and whether teams have a measurable and shared goal.

“These may refer to school organizational characteristics, such as the role of the school leader (e.g., facilitation, encouragement) (Knapp, Copland, & Swinnerton, 2007;Knapp, Swinnerton, Copland, & Monpas-Huber, 2006), and the degree to which teams have a measurable and shared goal (Datnow et al., 2007;Earl & Katz, 2006;Kerr et al., 2006;Wayman & Stringfield, 2006;Wohlstetter et al., 2008).”

• cites Knapp, Copland, & Swinnerton, 2007Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Datnow et al., 2007Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Earl & Katz, 2006Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.