Toward Automated Feedback on Teacher Discourse to Enhance Teacher Learning
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Like anyone, teachers need feedback to improve. Due to the high cost of human classroom observation, teachers receive infrequent feedback which is often more focused on evaluating performance than on improving practice. To address this critical barrier to teacher learning, we aim to provide teachers with detailed and actionable automated feedback. Towards this end, we developed an approach that enables teachers to easily record high-quality audio from their classes. Using this approach, teachers recorded 142 classroom sessions, of which 127 (89%) were usable. Next, we used speech recognition and machine learning to develop teacher-generalizable computer-scored estimates of key dimensions of teacher discourse. We found that automated models were moderately accurate when compared to human coders and that speech recognition errors did not influence performance. We conclude that authentic teacher discourse can be recorded and analyzed for automatic feedback. Our next step is to incorporate the automatic models into an interactive visualization tool that will provide teachers with objective feedback on the quality of their discourse.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
The role of deliberate practice in the acquisition of expert performance
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993) · Psychological Review · doi:10.1037/0033-295x.100.3.363 · Cited by 9,564 · cited 2×
“…for proficiency, achieving expertise requires deliberate practice, usually under the guidance of a coach, with carefully designed training tasks at an appropriate level of difficulty such that progress should be achievable in the short-term with effort, feedback, and guidance [19, 21, 22].”
BackgroundShute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631
Professional Development and Teacher Learning: Mapping the Terrain
Borko, H. (2004) · Educational Researcher · doi:10.3102/0013189x033008003 · Cited by 4,208 · cited 1×
“These findings confirm the results of two federally-funded studies showing that when decontextualized from the classroom, traditional professional development has failed to make an impact on teacher performance or student outcomes [8, 27, 28].”
ResultComputing inter‐rater reliability and its variance in the presence of high agreement
Gwet, K. L. (2008) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711006x126600 · Cited by 2,167 · cited 1×
“Coders reached a reliability threshold of 80% (using Gwet’s AC metric [32]) across all discourse variables prior to independent coding.”
MethodEricsson, K. A. (2006) · The Cambridge Handbook of Expertise and Expert Performance · doi:10.1017/cbo9780511816796.038 · Cited by 1,629 · cited 1×
“…for proficiency, achieving expertise requires deliberate practice, usually under the guidance of a coach, with carefully designed training tasks at an appropriate level of difficulty such that progress should be achievable in the short-term with effort, feedback, and guidance [19, 21, 22].”
BackgroundStudent engagement in high school classrooms from the perspective of flow theory
Shernoff, D. J., Csikszentmihalyi, M., Shneider, B., & Shernoff, E. S. (2003) · School Psychology Quarterly · doi:10.1521/scpq.18.2.158.21860 · Cited by 1,237 · cited 1×
“In order to move beyond the study of interactive discourse occurring in question and answer sessions—a guiding empirical component of prior work—our theoretical framework incorporates insights from research by Shernoff and colleagues [61, 62] and Grossman et al.”
MethodWord association norms, mutual information, and lexicography
Church, K. W., & Hanks, P. (1989) · Proceedings of the 27th annual meeting on Association for Computational Linguistics - · doi:10.3115/981623.981633 · Cited by 1,094
Hill, H. C., Blunk, M. L., Charalambous, C. Y., Lewis, J. M., Phelps, G. C., Sleep, L., & Ball, D. L. (2008) · Cognition and Instruction · doi:10.1080/07370000802177235 · Cited by 996 · cited 1×
“[45, 61], use of ELA terms [20, 33], cognitive level [9, 30, 53, 69], and elaborated feedback.”
BackgroundThe INTERSPEECH 2009 emotion challenge
Schuller, B., Steidl, S., & Batliner, A. (2009) · Interspeech 2009 · doi:10.21437/interspeech.2009-103 · Cited by 938 · cited 1×
“First, we extracted acoustic features using OpenSmile [23], using a standard feature set from the 2009 Interspeech Emotion challenge [59].”
MethodExamining the effects of classroom discussion on students’ comprehension of text: A meta-analysis.
Murphy, P. K., Wilkinson, I. A. G., Soter, A. O., Hennessey, M. N., & Alexander, J. F. (2009) · Journal of Educational Psychology · doi:10.1037/a0015576 · Cited by 566 · cited 1×
“Thus, the theoretical framework employed in the present study builds very closely on this program of research on dialogically organized instruction, emphasizing questions , authentic questions , and uptake amongst other dialogic discourse variables (see also [37, 44]).”
BackgroundDeliberate Practice Spells Success: Why Grittier Competitors Triumph at the National Spelling Bee
Duckworth, A. L., Kirby, T. A., Tsukayama, E., Berstein, H., & Ericsson, K. A. (2011) · Social Psychological and Personality Science · doi:10.1177/1948550610385872 · Cited by 560 · cited 1×
“…for proficiency, achieving expertise requires deliberate practice, usually under the guidance of a coach, with carefully designed training tasks at an appropriate level of difficulty such that progress should be achievable in the short-term with effort, feedback, and guidance [19, 21, 22].”
BackgroundA Meta-Analysis of the Effects of Feedback in Computer-Based Instruction
Azevedo, R., & Bernard, R. M. (1995) · Journal of Educational Computing Research · doi:10.2190/9lmd-3u28-3a0g-ftqt · Cited by 308
Student Learning Benefits of a Mixed-Reality Teacher Awareness Tool in AI-Enhanced Classrooms
Holstein, K., McLaren, B. M., & Aleven, V. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93843-1_12 · Cited by 235
Automatically Measuring Question Authenticity in Real-World Classrooms
Kelly, S., Olney, A. M., Donnelly, P., Nystrand, M., & D’Mello, S. K. (2018) · Educational Researcher · doi:10.3102/0013189x18785613 · Cited by 114 · cited 2×
“This approach has been promising when predicting the prevalence of infrequent discourse strategies such as open-ended questions [38].”
BackgroundSelf-citationTeacher Dashboards in Practice: Usage and Impact
Molenaar, I., & Knoop-van Campen, C. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66610-5_10 · Cited by 68
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Dimitriadou, E., & Lanitis, A. (2023) · Smart Learning Environments · doi:10.1186/s40561-023-00231-3 · Cited by 370
“Jensen et al. (2020) devised a method for teachers to effortlessly audiotape the conversations and lectures in a classroom.”
MethodArtificial intelligence and learning analytics in teacher education: A systematic review
Salas-Pilco, S., Xiao, K., & Hu, X. (2022) · Education Sciences · doi:10.3390/educsci12080569 · Cited by 326
“However, research is also highlighting some ethical concerns, such as privacy when compromising the exploitation of data via recommender systems [77], tracking systems that gather detailed information about actions and preferences [78], and bias and discrimination, e.g., perpetuating gender bias…”
BackgroundHahn, M. G., Navarro, S. M. B., De La Fuente Valentin, L., & Burgos, D. (2021) · IEEE Access · doi:10.1109/access.2021.3100890 · Cited by 111
“Other potential adverse effects included loss of human interaction and lack of interpersonal skills while solving problems [64], and lack of personalized feedback that could help outlier students [68], especially struggling students [69].”
BackgroundWang, R. E., & Demszky, D. (2023) · arXiv · doi:10.48550/arxiv.2306.03090 · Cited by 74
“Recent NLP advances are able to provide teachers feedback on their classroom discourse, promoting selfreflection and instructional development (Samei et al., 2014; Donnelly et al., 2017; Kelly et al., 2018; Jensen et al., 2020).”
Background
Metadata
- Authors
- Jensen, E., Dale, M., Donnelly, P. J., Stone, C., Kelly, S., Godley, A., & D'Mello, S. K.
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-13
- OpenAlex
- W3028958483
- Cited by
- 90
- References
- 16
- FWCI
- 13.68
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.