Making Sense of Data-Driven Decision Making in Education: Evidence from Recent RAND Research
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index; none of the 35 sources its claims cite is checked for support yet.
Abstract
Drawing on RAND research, this paper examines data-driven decision making (DDDM) policies and practices, clarifies how data are being used in schools and districts, identifies factors that support or hinder DDDM, and suggests future research in the field. A few implications are that schools and teachers need support to implement effective DDDM and that utilizing multiple data types collected at multiple time points can improve decision making.
Overview
CoChat Summary
TL;DR
This RAND occasional paper synthesizes findings from four recent RAND studies to describe how educators collect and use data for decision making, what supports exist, and what factors shape use. It finds that outcome data—especially state test scores—dominate practice, while input, process, and satisfaction data are used far less systematically, and that DDDM does not guarantee better decisions.
Key findings
- Achievement test scores receive the most systematic attention across sites; the vast majority of superintendents, principals, and teachers surveyed use them, and more than 80 percent of superintendents in California, Georgia, and Pennsylvania found local assessment results more useful for decision making than state test results.
- Local interim progress tests are growing in popularity: 89 percent of Georgia districts require mathematics progress tests (half require them in science), roughly one-half of California districts and one-third of Pennsylvania districts require mathematics progress tests, and about 80 percent of principals in one Institute for Learning (IFL) district rated progress-test results moderately to very useful for instructional decisions.
- A common use of test scores is identifying "bubble kids" near proficiency cutoffs: more than three-quarters of principals in all three states reported their school or district encourages teachers to focus on these students, while between one-quarter and one-third of teachers said they actually do.
- Use varies more within schools than between them—approximately 80 percent or more of the variability in teacher survey reports of several forms of data use in the ISBA project was within rather than between schools—and process, input, and satisfaction data were used less frequently than outcome data.
Methods
- Synthesis of four RAND studies: Implementing Standards-Based Accountability (ISBA, 2002–2007, statewide samples in California, Georgia, Pennsylvania), Data-driven decision making in Southwestern Pennsylvania (SWPA, 2004–2005, 6 districts and 1 charter school), Instructional improvement efforts of districts partnered with the Institute for Learning (IFL, 2002–2005, 3 urban districts), and Evaluation of Edison Schools (2000–2005, 23 schools).
- Data sources across the studies included surveys of superintendents, principals, and teachers; interviews and focus groups; observations of trainings and meetings; and document reviews (e.g., training materials, school improvement plans, data-use tools).
- The paper organizes findings around four questions—what data are used, how they are used, what support exists, and what factors influence use—using a conceptual framework adapted from the literature (Mandinach, Honey, and Light, 2006).
Limitations
- Stated by the paper: the studies were primarily descriptive and do not address the effects of DDDM on student outcomes; they were not deliberately designed to collect data from a representative sample of U.S. school districts; they did not capture all data used by educators or all influences on decision making, so evidence on test data is more extensive than on other data forms.
- Stated by the paper: general awareness and understanding of value-added modeling among principals and teachers appear to be quite low, and low survey response rates historically rendered Edison's satisfaction data of limited value.
- Evident from the text: findings are drawn from a small number of purposively selected sites (three states, six SWPA districts, three IFL districts, 23 Edison schools), so cross-site comparisons and generalizations beyond these contexts are constrained.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 4 Not valid · 20 Not checked · 154 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For this reason and others, many districts and schools have adopted formal local tests, given more frequently throughout the year and providing diagnostic information that could be acted on immediately.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our research also suggests that administrators perceive these systems of local progress tests as powerful tools for school improvement-particularly when compared to state tests.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“More than 80 percent of superintendents in California, Georgia, and Pennsylvania found results from local assessments to be more useful for decision making than state test results.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Approximately one-half of California districts and one-third of Pennsylvania districts require mathematics progress tests in some or all of their schools.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, approximately 80 percent of principals in one IFL district that implemented standards-aligned progress tests reported that these results were moderately to very useful for guiding decisions about instruction.”
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
101 more claims
Create a free account to see all 178 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 35
- It may state two separate claims as one. 46
Citations
Cites
The fifth discipline, the art and practice of the learning organization
Senge, P. M. (1991) · Performance + Instruction · doi:10.1002/pfi.4170300510 · Cited by 20,060
Assessment and Classroom Learning
Black, P., & Wiliam, D. (1998) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969595980050102 · Cited by 8,152
Effective Schools for the Urban Poor
Edmonds, R. R. (1979) · Educational leadership · Cited by 1,912
Below the Bubble: “Educational Triage” and the Texas Accountability System
Booher-Jennings, J. (2005) · American Educational Research Journal · doi:10.3102/00028312042002231 · Cited by 762
Leadership of Inquiry: Building and Sustaining Capacity for School Improvement
Copland, M. A. (2003) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737025004375 · Cited by 582
Evaluating Value-Added Models for Teacher Accountability
McCaffrey, D. F., Lockwood, J. R., Koretz, D. M., & Hamilton, L. S. (2003) · PsycEXTRA Dataset · doi:10.1037/e658712010-001 · Cited by 541
Self-citationAccountability Policies and Teacher Decision Making: Barriers to the Use of Data to Improve Practice
Ingram, D., Louis, K. S., & Schroeder, R. G. (2004) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1111/j.1467-9620.2004.00379.x · Cited by 373
Spillane, J. P., & Thompson, C. L. (1997) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737019002185 · Cited by 372
(2003) · Cited by 310
The Merits of Measurement-Driven Instruction.
Popham, W. J. (1987) · Phi Delta Kappan · Cited by 300
Research works printed but not linked: 7
- Choppin, Jeffrey, 'Data Use in Practice: Examples from the School Level,' paper presented at the Annual Conference of the American Educational Research Association, New Orleans, La., April 2002.
- Coburn, C., M. I. Honig, and M. K. Stein, What's the Evidence on Districts' Use of Evidence? chapter prepared for conference volume, sponsored by the MacArthur Network on Teaching and Learning, 2005.
- Deming, W. E., Out of Crisis , Cambridge, Mass.: MIT Center for Advanced Engineering Study, 1986.
- Juran, J. M., On Planning for Quality , New York: Free Press, 1988.
- Keeney, Lorraine, Using Data for School Improvement: Report on the Second Practitioners' Conference for Annenberg Challenge Sites , Houston, Tex., May 1998.
- National Council on Measurement in Education (NCME), Newsletter, Vol. 13, No. 3, September 2005.
- Olson, L. 'ETS to Enter Formative-Assessment Market at K-12 Level,' Education Week , Vol. 24, No. 25, p. 11, March 2, 2005.
Cited by
Items 201–202 of 202
Applying Advanced Data Science Tools to Enhance University Teacher Evaluation Processes
Meza, A. R. J., Vargas, M. P. M., Pérez, W. D., et al. (2026) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-032-13703-6_12
DORA-II Technical Adequacy Brief: Measuring the Process and Outcomes of Team Problem Solving.
Algozzine, B., Horner, R. H., Todd, A. W., Newton, J. S., Algozzine, K. M., & Cusumano, D. L. (2014) · Grantee Submission
Metadata
- Authors
- Marsh, J. A., Pane, J. F., & Hamilton, L. S.
- Venue
- RAND Corporation eBooks
- Publisher
- RAND Corporation
- Published
- 2006
- OpenAlex
- W2995703487
- Cited by
- 202
- References
- 33
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 8 · 2023: 9 · 2022: 15 · 2021: 13 · 2020: 15 · 2019: 11 · 2018: 8 · 2017: 8 · 2016: 16 · 2015: 20 · 2014: 17 · 2013: 14 · 2012: 13
- FWCI
- 14.20
- Type
- book
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- RAND Corporation eBooks (publishedVersion)
- Topics
- Social Sciences › Education › Educational Assessment and Improvement
- Social Sciences › Education › School Choice and Performance
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Keywords
- data-driven decision making, education policy, data use in schools, school districts, teacher support, longitudinal data, barriers to data use, educational decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.