Online interventions to promote teacher data-driven decision making: Optimizing design to maximize impact
The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
Although most schools are steeped in data, many teachers have not had adequate opportunities to learn how to analyze, interpret, and instructionally use data. This paper reports on the efficacy of a suite of online, data literacy interventions for in-service (n = 25) and pre-service teachers (n = 99). The paper also tests the impact of including pedagogical knowledge and data interpretational scaffolds in such interventions. Findings from two implementations of the interventions indicate changes in in-service and pre-service teachers’ data-driven decision making self-efficacy (ds ranged from 0.54 to 0.98) and anxiety (ds were −0.62 and −0.71 respectively), and pre-service teachers’ assessment beliefs (ds ranged from 0.23 to 0.62). However, results imply limited value-added effects of scaffolds on the considered outcome variables. Implications for teacher education to support teacher data literacy and data-driven decision making are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449
Schafer, J. L. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800102 · Cited by 3,949
Teaching Courses Online: A Review of the Research
Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S. M., & Liu, X. (2006) · Review of Educational Research · doi:10.3102/00346543076001093 · Cited by 1,267
Self-efficacy conception of anxiety
Bandura, A. (1988) · Anxiety Research · doi:10.1080/10615808808248222 · Cited by 801
Handbook of Human and Social Conditions in Assessment
(2016) · doi:10.4324/9781315749136 · Cited by 447
Accountability Policies and Teacher Decision Making: Barriers to the Use of Data to Improve Practice
Ingram, D., Louis, K. S., & Schroeder, R. G. (2004) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1111/j.1467-9620.2004.00379.x · Cited by 373
Mandinach, E. B., & Gummer, E. S. (2016) · Teaching and Teacher Education · doi:10.1016/j.tate.2016.07.011 · Cited by 363
Interventions Promoting Educators’ Use of Data: Research Insights and Gaps
Marsh, J. A. (2012) · Teachers College Record: The Voice of Scholarship in Education · doi:10.1177/016146811211401106 · Cited by 328
Datnow, A., & Hubbard, L. (2016) · Journal of Educational Change · doi:10.1007/s10833-015-9264-2 · Cited by 287
Teachers’ Use of Data: Loose Coupling, Agenda Setting, and Team Norms
Young, V. M. (2006) · American Journal of Education · doi:10.1086/505058 · Cited by 281
Carlson, D., Borman, G. D., & Robinson, M. (2011) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373711412765 · Cited by 249
Creating a system for data-driven decision-making: applying the principal-agent framework
Wohlstetter, P., Datnow, A., & Park, V. (2008) · School Effectiveness and School Improvement · doi:10.1080/09243450802246376 · Cited by 248
Data teams for school improvement
Schildkamp, K., Poortman, C. L., & Handelzalts, A. (2016) · School Effectiveness and School Improvement · doi:10.1080/09243453.2015.1056192 · Cited by 229
Marsh, J. A., Sloan McCombs, J., & Martorell, F. (2010) · Educational Policy · doi:10.1177/0895904809341467 · Cited by 181
Exploring data use practices around Europe: Identifying enablers and barriers
Schildkamp, K., Karbautzki, L., & Vanhoof, J. (2014) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2013.10.007 · Cited by 168
Hoogland, I., Schildkamp, K., van der Kleij, F., Heitink, M., Kippers, W., Veldkamp, B., & Dijkstra, A. M. (2016) · Teaching and Teacher Education · doi:10.1016/j.tate.2016.07.012 · Cited by 158
Prenger, R., & Schildkamp, K. (2018) · Educational Psychology · doi:10.1080/01443410.2018.1426834 · Cited by 101
Examining the landscape of teacher learning for data use: The case of Illinois
Reeves, T. D., Summers, K. H., & Grove, E. (2016) · Cogent Education · doi:10.1080/2331186x.2016.1211476 · Cited by 35
Self-citation
Research works printed but not linked: 41
- Aydin, 2011, Interplay of motivational and cognitive strategies in predicting self-efficacy and anxiety, Educational Psychology, 31, 55, 10.1080/01443410.2010.518561 · DOI
- Bocala, 2015, Teaching educators habits of mind for using data wisely, Teachers College Record, 117, 1, 10.1177/016146811511700409 · DOI · Paper page
- Brown, 2006, Teachers’ conceptions of assessment: Validation of an abridged version, Psychological Reports, 99, 166, 10.2466/pr0.99.1.166-170 · DOI · Paper page
- Chen, 2009, The use of online synchronous discussion for web-based professional development for teachers, Computers & Education, 53, 1155, 10.1016/j.compedu.2009.05.026 · DOI · Paper page
- Chick, 2013, The statistical literacy needed to interpret school assessment data, Mathematics Teacher Education and Development, 15, 5
- Chou, 2003, Interactivity and interactive functions in web‐based learning systems: A technical framework for designers, British Journal of Educational Technology, 34, 265, 10.1111/1467-8535.00326 · DOI · Paper page
- Coburn, 2011, Research on data use: A framework and analysis, Measurement Interdisciplinary Research and Perspectives, 9, 173, 10.1080/15366367.2011.626729 · DOI · Paper page
- Cowie, 2017, Exploring the challenge of developing student teacher data literacy, Assessment in Education Principles Policy and Practice, 24, 147, 10.1080/0969594X.2016.1225668 · DOI
- Dede, 2009, A research agenda for online teacher professional development, Journal of Teacher Education, 60, 8, 10.1177/0022487108327554 · DOI · Paper page
- Dunn, 2016, Educational psychology’s instructional challenge: Pre-service teacher concerns regarding classroom-level data-driven decision-making, Psychology Learning & Teaching, 15, 31, 10.1177/1475725716636975 · DOI
- Dunn, 2013, What teachers think about what they can do with data: Development and validation of the data driven decision-making efficacy and anxiety inventory, Contemporary Educational Psychology, 38, 87, 10.1016/j.cedpsych.2012.11.002 · DOI
- Farrell, 2016, Contributing conditions: A qualitative comparative analysis of teachers’ instructional responses to data, Teaching and Teacher Education, 60, 398, 10.1016/j.tate.2016.07.010 · DOI
- Green, 2016, Teachers and data literacy: A blueprint for professional development to foster data driven decision making, Journal of Continuing Education and Professional Development, 3, 14
- Hoover, 2013, Teachers’ instructional use of summative student assessment data, Applied Measurement in Education, 26, 219, 10.1080/08957347.2013.793187 · DOI · Paper page
- Huguet, 2014, Building teachers’ data-use capacity: Insights from strong and developing coaches, Education Policy Analysis Archives, 22, 10.14507/epaa.v22n52.2014 · DOI · Paper page
- Jimerson, 2015, Supporting data-informed practice among early career teachers: The role of mentors, Leadership and Policy in Schools, 14, 204, 10.1080/15700763.2014.997937 · DOI
- Lai, 2016, The impact of data use professional development on student achievement, Teaching and Teacher Education, 60, 434, 10.1016/j.tate.2016.07.005 · DOI
- Little, 1988, A test of missing completely at random for multivariate data with missing values, Journal of the American Statistical Association, 83, 1198, 10.1080/01621459.1988.10478722 · DOI · Paper page
- Mandinach, 2016, Teachers learning how to use data: A synthesis of the issues and what is known, Teaching and Teacher Education, 60, 452, 10.1016/j.tate.2016.07.009 · DOI
- Marsh, 2015, How leaders can support teachers with data-driven decision making: A framework for understanding capacity building, Educational Management Administration & Leadership, 43, 269, 10.1177/1741143214537229 · DOI · Paper page
- Marsh, 2015, Using data to alter instructional practice: The mediating role of coaches and professional learning communities, Teachers College Record, 117, 1, 10.1177/016146811511700411 · DOI · Paper page
- Moore, 1989, Three types of interaction, The American Journal of Distance Education, 3, 1, 10.1080/08923648909526659 · DOI · Paper page
- Neuman, 2016, Code red: The danger of data-driven instruction, Educational Leadership, 74, 24
- Oláh, 2010, Learning to learn from benchmark assessment data: How teachers analyze results, Peabody Journal of Education, 85, 226, 10.1080/01619561003688688 · DOI · Paper page
- Pajares, 1992, Teachers’ beliefs and educational research: Cleaning up a messy construct, Review of Educational Research, 62, 307, 10.3102/00346543062003307 · DOI · Paper page
- Pierce, 2013, Workplace statistical literacy for teachers: Interpreting box plots, Mathematics Education Research Journal, 25, 189, 10.1007/s13394-012-0046-3 · DOI
- Pierce, 2013, Teachers’ perceptions of the factors influencing their engagement with statistical reports on student achievement data, Australian Journal of Education, 57, 237, 10.1177/0004944113496176 · DOI
- Pierce, 2014, A statistical literacy hierarchy for interpreting educational system data, Australian Journal of Education, 58, 195, 10.1177/0004944114530067 · DOI
- Reeves, 2011, Predictors of teacher satisfaction with online professional development: Evidence from the e-Learning for Educators (EfE) initiative, Prof. Dev. Educ., 37, 591, 10.1080/19415257.2011.553824 · DOI · Paper page
- Reeves, 2017, Building pre-service teacher capacity to use external assessment data: An intervention study, Teach. Educ., 52, 155, 10.1080/08878730.2016.1273420 · DOI
- Reeves, T. D., Chiang, J. L. (in review). Effects of an asynchronous online data literacy intervention on pre-service and in-service teachers’ beliefs, self-efficacy, and practices. Unpublished manuscript.
- Rolando, 2014, Learning with their peers: Using a virtual learning community to improve an in-service Biology teacher education program in Brazil, Teaching and Teacher Education, 44, 44, 10.1016/j.tate.2014.07.010 · DOI
- Shadish, 2002, Experimental and quasi-experimental designs for generalized causal inference, Belmont, CA: Wadsworth Cengage Learning.
- Sun, 2016, A review of research on teachers’ use of student data: From the perspective of school leadership, Educational Assessment Evaluation and Accountability, 28, 5, 10.1007/s11092-016-9238-9 · DOI
- Turner, 2012, The practice of data use: An introduction, American Journal of Education, 118, 99, 10.1086/663272 · DOI · Paper page
- Van den Hurk, 2016, Fostering effective teaching behavior through the use of data-feedback, Teaching and Teacher Education, 60, 444, 10.1016/j.tate.2016.07.003 · DOI
- Van Gasse, 2016, Teacher collaboration on the use of pupil learning outcome data: A rich environment for professional learning?, Teaching and Teacher Education, 60, 387, 10.1016/j.tate.2016.07.004 · DOI
- van Geel, 2016, Assessing the effects of a school-wide data-based decision-making intervention on student achievement growth in primary schools, American Educational Research Journal, 53, 360, 10.3102/0002831216637346 · DOI · Paper page
- Vanhoof, 2014, From ‘professional development for data use’ to ‘data use for professional development’, Studies in Educational Evaluation, 42, 1, 10.1016/j.stueduc.2014.05.001 · DOI
- Volante, 2007, Exploring teacher candidates’ assessment literacy: Implications for teacher education reform and professional development, Canadian Journal of Education, 30, 749, 10.2307/20466661 · DOI · Paper page
- Wayman, 2014, Teacher needs for data-related professional learning, Studies in Educational Evaluation, 42, 25, 10.1016/j.stueduc.2013.11.001 · DOI
Cited by
Improving writing feedback for struggling writers: generative AI to the rescue?
Evmenova, A. S., Regan, K., Mergen, R., & Hrisseh, R. (2024) · TechTrends · doi:10.1007/s11528-024-00965-y · Cited by 51
“After the student has an opportunity to write independently, the teacher can engage in what we refer to as data-driven decision making (or DDDM; Park & Datnow, 2017; Reeves and Chiang, 2018).”
A continuum of data literacy for teaching
Beck, J. S., & Nunnaley, D. (2021) · Studies in Educational Evaluation · doi:10.1016/j.stueduc.2020.100871 · Cited by 44
Ansyari, M. F., Groot, W., & De Witte, K. (2020) · Educational Research Review · doi:10.1016/j.edurev.2020.100362 · Cited by 26
“…different kinds of questions (e.g., achievement status and growth, strengths and weaknesses) at five different M.F. Ansyari et al. student levels (e.g., individual, subgroup, school) with external, standardised assessment data presented in tables, charts, and score reports (Reeves & Chiang, 2018).”
BackgroundMethodResultInfluentialAtai-Tabar, M., Zareian, G., Amirian, S. M. R., & Adel, S. M. R. (2024) · Journal of Applied Research in Higher Education · doi:10.1108/jarhe-04-2023-0169 · Cited by 4
Metadata
- Authors
- Reeves, T. D., & Chiang, J.-L.
- Venue
- Studies in Educational Evaluation
- Publisher
- Elsevier BV
- Published
- 2018-12
- Volume
- 59
- Pages
- 256-269
- ISSN
- 0191-491X (ISSN-L)
- OpenAlex
- W2897089626
- Cited by
- 44
- References
- 18
- FWCI
- 4.22
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- Funders
- College of Education (COE)
- Proposal or Pilot Grant from the Center for the Interdisciplinary Study of Language and Literacy (CISLL)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.