CoChat
Sign Up
Cited by 0Educational Technology and PedagogyEvaluation of Teaching PracticesOnline Learning and Analytics

AI-Assisted Classroom Interaction Analysis for Enhancing Teaching Effectiveness in Higher Education

Open access
Wang, Yanping·2026·Proceedings of the 2026 International Conference on Artificial Intelligence, Digital Media and Education
DOI 10.1145/3839592.3839666
Open this paper in CoChat — free accountOpen PDF

Abstract

Classroom interaction is a fundamental determinant of learning quality in higher education. Conventional approaches to assessing teaching effectiveness—peer review, end-of-semester student surveys, and periodic classroom observations—are retrospective, inconsistent, and do not scale to large faculty populations.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper presents CIIS, a multimodal AI framework integrating ASR, NLP, computer vision, and machine learning for real-time classroom interaction analysis and instructor feedback in higher education. A semester-long quasi-experimental pilot across six courses (n = 247) reports significant improvements in engagement, teaching effectiveness, satisfaction, and learning outcomes, alongside 91.3% discourse classification accuracy and a 68% improvement in instructor responsiveness to low-engagement alerts.

Key findings

  • CIIS technical performance: discourse classification accuracy 91.3% (macro F1 = 0.887), engagement detector correlation r = 0.84 with human ratings, ASR word error rate 6.1%, mean processing latency 1.8 s per 10-s window, and inter-rater reliability κ = 0.81.
  • Intervention courses improved significantly relative to controls on all four effectiveness measures (p < 0.01): TEQ-20 increased by 14.2 points (Cohen’s d = 1.21) versus 3.1 points for controls; SEI-HE increased by 11.7 points versus 2.4; CSS rose from 3.6 to 4.3 versus 3.7 to 3.9; ALO rose from 67.2% to 73.8% versus 66.8% to 68.9%.
  • High-engagement windows (Q4) versus low-engagement windows (Q1) showed shorter instructor monologues (1.3 vs 4.7 min), more student turns (8.9 vs 1.4 per 10 min), more open-ended questions (57% vs 12%), higher facial affect scores (0.78 vs 0.31), and more front-directed gaze (79% vs 41%).
  • Intervention instructors changed behaviour: mean uninterrupted lecture duration fell from 5.1 to 2.4 min (−53%; paired t(5) = 6.47, p = 0.001), open-ended questions per 50-min session rose from 2.3 to 7.8, and mean alert response time fell from 4.3 to 1.4 min (68% improvement); dashboard access occurred in a mean of 94% of monitored sessions (range 89%–97%).

Methods

  • Quasi-experimental, two-condition design across one semester at the University of Tehran and Tsinghua University; six purposively sampled courses in computer science, social sciences, and natural sciences were assigned to intervention (CIIS-enabled feedback) or control (standard instruction), with n = 247 and no course-level randomisation due to institutional scheduling constraints.
  • Teaching effectiveness was measured with SEI-HE (35-item Likert), TEQ-20, CSS, and ALO from final examinations; SEI-HE and TEQ-20 were administered pre- and post-semester, while CSS and ALO were collected post-semester only.
  • Technical evaluation used 48 hours of manually annotated classroom video (8 hours per course) with two trained human coders; analyses included classification accuracy, macro F1, engagement correlation, ASR word error rate, processing latency, and Cohen’s kappa.
  • CIIS comprises four layers: data acquisition (HD cameras, eight-channel beamforming microphone array, LMS/clicker feeds), multimodal feature extraction (Whisper-large ASR, BERT-based classifier, OpenFace 2.0 facial action units, gaze/head-pose, prosody), interaction classification (late-fusion ensemble, 30-s windows, eight interaction classes, Engagement Index 0–1, SessionState updated every 10 s), and feedback generation (real-time dashboard, session reports, longitudinal analytics, recommendations).

Limitations

  • Stated: the study involved six courses at two institutions, restricting generalisability across disciplines and educational cultures; the quasi-experimental design precludes strong causal inference and an RCT would be required; system performance was evaluated under controlled hardware conditions and may degrade with suboptimal acoustics or lighting; sustainability of instructor behavioural changes beyond the study period is unknown and longitudinal follow-up is required.
  • Evident from the text: only three intervention courses/instructors generated the instructor behavioural adaptation and dashboard-use findings, so those instructor-level results rest on a very small sample; course-level assignment was not randomised and conditions are confounded with course, discipline, and institution.
  • Evident from the text: technical validation is limited to 48 hours of annotated video and two human coders, and the study reports no long-term follow-up beyond the semester.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3839592.3839666

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

3 of 8 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured

  • Premisecitation
    § 1 INTRODUCTION · p. 1

    The systematic deployment of AI-powered classroom analytics in higher education remains limited.

    “However, the systematic deployment of AI-powered classroom analytics in higher education remains limited.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [4] [4] D. Gašević, S. Dawson, and G. Siemens, 'Let's not forget: Learning analytics are about learning,' TechTrends, vol. 59, no. 1, pp. 64-71, Jan. 2015, doi: 10.1007/s11528-014-0822-x. - source

      The available indexed text (abstract; no page given) describes learning analytics as an emerging field and identifies challenges to its future impact. Searches for systematic deployment, classroom use, and AI-powered analytics found no passage assessing how widely such systems have been deployed in higher education. The available copy contains only abstract and bibliographic material, so the claim cannot be verified from it. (LLM, conf=0.72)

      Experimental — accuracy not yet measured
  • Findingown data
    § 5.2 Teaching Effectiveness Outcomes · p. 4

    The largest effect was on the TEQ-20: intervention instructors improved by a mean of 14.2 points (Cohen's d = 1.21) compared with 3.1 points for controls.

    “The largest effect was observed on the TEQ-20: intervention instructors improved by a mean of 14.2 points (Cohen's d = 1.21) compared with 3.1 points for controls.”
    Quote in the paperSubstantial claim
  • Findingown data
    § 5.3 Interaction Pattern Analysis · p. 4

    Low-engagement windows (Q1) showed extended uninterrupted instructor speech (mean 4.7 min).

    “Low-engagement windows (Q1) showed extended uninterrupted instructor speech (mean 4.7 min)”
    Quote in the paperSubstantial claim
  • Findingown data
    § 5.3 Interaction Pattern Analysis · p. 4

    Low-engagement windows (Q1) showed minimal student verbal participation (1.4 turns per 10 min).

    “minimal student verbal participation (1.4 turns per 10 min)”
    Quote in the paperSubstantial claim
  • Findingown data
    § 5.3 Interaction Pattern Analysis · p. 4

    High-engagement windows (Q4) had more frequent front-directed gaze (79%).

    “more frequent front-directed gaze (79%)”
    Quote in the paperSubstantial claim
  • May combine two claims. 13
  • The quote only partly states this claim. 1
  • Classified as probably not a substantial claim. 4

  • May combine two claims.

    Premisecitation
    § 1 INTRODUCTION · p. 1

    Key challenges to deploying AI-powered classroom analytics include privacy concerns, variability across teaching styles and disciplines, computational requirements for real-time processing, and the absence of validated frameworks for translating raw interaction data into actionable pedagogical guidance.

    “Key challenges include privacy concerns, variability across teaching styles and disciplines, computational requirements for real-time processing, and the absence of validated frameworks for translating raw interaction data into actionable pedagogical guidance [4].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [4] [4] D. Gašević, S. Dawson, and G. Siemens, 'Let's not forget: Learning analytics are about learning,' TechTrends, vol. 59, no. 1, pp. 64-71, Jan. 2015, doi: 10.1007/s11528-014-0822-x. - source

      The indexed passage labeled AB (no page given) says that learning analytics faces critical research issues and that computational work should be integrated with educational research. It does not identify the cited sentence’s specific challenges. Searches for privacy, teaching-style and disciplinary variability, real-time processing, and validated frameworks returned no passage addressing them. (LLM, conf=0.73)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation
    § 1 INTRODUCTION · p. 1

    Modern AI systems can accurately transcribe speech, detect affective states from facial expressions, estimate student attention from gaze direction, and classify discourse types from linguistic features.

    “Modern AI systems can accurately transcribe speech, detect affective states from facial expressions, estimate student attention from gaze direction, and classify discourse types from linguistic features-capabilities that, when combined, offer unprecedented insight into the structure and quality of classroom interaction [2], [3].”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [2] [2] M. Nazaretsky, M. Ariely, M. Cukurova, and G. Alexandron, 'Teachers' trust in AI-powered educational technology and a professional development program to improve it,' Brit. J. Educational Technol., vol. 53, no. 4, pp. 914-931, Jul. 2022, doi: 10.1111/bjet.13232. - source

      The Data collection and discourse analysis section describes researchers classifying teachers’ utterances; it does not describe AI classifying classroom discourse. Searches for speech transcription, facial affect detection, gaze-based attention estimates, and classroom-interaction analysis found no passage addressing the claimed capabilities. Page numbers were not available in our copy. (LLM, conf=0.77)

      Experimental — accuracy not yet measured
    • cites [3] [3] L. Chen, P. Chen, and Z. Lin, 'Artificial intelligence in education: A review,' IEEE Access, vol. 10, pp. 75331-75346, 2022, doi: 10.1109/ACCESS.2022.3191498. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data
    § 5.4 Instructor Behavioural Adaptation · p. 5

    Mean response time to low-engagement alerts decreased from 4.3 min during weeks 1-2 to 1.4 min during weeks 13-14, representing a 68% improvement in instructor responsiveness.

    “Mean response time to low-engagement alerts decreased from 4.3 min during weeks 1-2 to 1.4 min during weeks 13-14, representing a 68% improvement in instructor responsiveness.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.1 System Performance · p. 3

    The discourse classifier achieved an overall accuracy of 91.3% (macro F1 = 0.887).

    “The discourse classifier achieved an overall accuracy of 91.3% (macro F1 = 0.887);”
    • Also stated in: § 7 CONCLUSION · p. 6 · plain
      “The system achieved a discourse classification accuracy of 91.3%”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § 1 INTRODUCTION · p. 1

    All intervention courses showed statistically significant improvements relative to controls on all four teaching effectiveness measures (p < 0.01).

    “we report empirical results from a semester-long quasi-experimental pilot study across six courses at two universities (n = 247), demonstrating statistically significant improvements in teaching effectiveness indicators.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given
    § 4.1 Study Design · p. 3

    Written informed consent was obtained from all participants, and ethical approval was granted by the Institutional Review Boards of both universities.

    “Written informed consent was obtained from all participants, and ethical approval was granted by the Institutional Review Boards of both universities.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data
    § 4.3 Technical Evaluation · p. 3

    System performance was evaluated against a manually annotated reference set comprising 48 hours of classroom video (8 hours per course) recorded at mid-semester.

    “System performance was evaluated against a manually annotated reference set comprising 48 hours of classroom video (8 hours per course) recorded at mid-semester.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation
    § 6.1 Interpretation of Findings · p. 5

    The 14.2-point gain on the TEQ-20 exceeds the minimal important difference (MID) of 8 points established in prior validation studies of the instrument, indicating an educationally meaningful effect.

    “The 14.2-point gain on the TEQ-20 exceeds the minimal important difference (MID) of 8 points established in prior validation studies of the instrument [12], indicating an educationally meaningful effect.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [12] [12] P. C. Abrami, C. A. Wade, V. Pillay, O. Aslan, and E. Bures, 'Interactions in distance education: A report from the field,' J. Distance Educ., vol. 22, no. 2, pp. 1-22, 2008. Available: https://www.ijede.ca/index.php/jde/issue/archive. -

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data
    § 6.1 Interpretation of Findings · p. 6

    Classrooms with shorter instructor monologues, higher student turn-taking rates, and more frequent open-ended questioning are associated with higher engagement.

    “The CIIS findings quantitatively corroborate long-standing qualitative claims in the educational literature: classrooms with shorter instructor monologues, higher student turn-taking rates, and more frequent open-ended questioning are associated with higher engagement.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given
    § 6.3 Limitations · p. 6

    System performance was evaluated under controlled hardware conditions and may degrade in classrooms with suboptimal acoustics or lighting.

    “System performance was evaluated under controlled hardware conditions and may degrade in classrooms with suboptimal acoustics or lighting.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § 7 CONCLUSION · p. 7

    These findings extend the evidence base for AI-supported pedagogical tools and demonstrate the potential of real-time, data-driven interaction feedback as a complement to retrospective evaluation methods.

    “These findings extend the evidence base for AI-supported pedagogical tools and demonstrate the potential of real-time, data-driven interaction feedback as a complement to retrospective evaluation methods.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductcitation
    § 5.1 System Performance · p. 3

    Inter-rater agreement among human annotators was κ = 0.81, indicating substantial agreement.

    “Inter-rater agreement among human annotators was 𝜅 = 0.81, indicating substantial agreement [14].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] [14] J. R. Landis and G. G. Koch, 'The measurement of observer agreement for categorical data, ' Biometrics, vol. 33, no. 1, pp. 159-174, Mar. 1977. doi: 10.2307/2529310. - source

  • May combine two claims.

    Study design and conductreasoning
    § 4.1 Study Design · p. 3

    Randomisation at the course level was not possible due to institutional scheduling constraints, which limits causal inference.

    “Randomisation at the course level was not possible due to institutional scheduling constraints, which limits causal inference and is acknowledged as a study limitation.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data
    § 4.3 Technical Evaluation · p. 3

    Two trained human coders independently labelled interaction categories and engagement levels; inter-rater reliability was quantified using Cohen's kappa.

    “Two trained human coders independently labelled interaction categories and engagement levels; inter-rater reliability was quantified using Cohen's kappa ( 𝜅 ).”
    Quote in the paperSubstantial claim
14 of 14

  • The paper relies on this without support.

    Conventional approaches to assessing teaching effectiveness are retrospective, inconsistent, and do not scale to large faculty populations.

    “Conventional approaches to assessing teaching effectiveness-peer review, end-of-semester student surveys, and periodic classroom observations-are retrospective, inconsistent, and do not scale to large faculty populations.”
    “These mechanisms are retrospective, coarse-grained, and provide no actionable real-time guidance to instructors seeking to improve in-session delivery.”

  • The paper relies on this without support.

    The SEI-HE, TEQ-20, CSS, and ALO are validated instruments for measuring teaching effectiveness.

    “Teaching effectiveness was assessed using four validated instruments”
2 of 2

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentt(5) = 6.47reported p = .001 · recomputed p = .001

    “t(5) = 6.47, p = .001”

1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Pre-semester scores reported for post-only measures

    Major

    The methods say course satisfaction (CSS) and learning outcomes (ALO) were collected only after the semester, but Table 3 reports pre-semester scores for both. This affects the reported pre-to-post improvements on those measures.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. 10.1109/access.2022.3191498 · DOI
  2. N. A. Flanders, Analysing Teaching Behaviour. Reading, MA, USA: Addison-Wesley, 1970. Available: https://books.google.com/books/about/Analyzing_Teaching_Behavior.html?id=SBSdAAAAMAAJ.
  3. 10.1109/tlt.2022.3223181 · DOI
  4. P. C. Abrami, C. A. Wade, V. Pillay, O. Aslan, and E. Bures, “Interactions in distance education: A report from the field,” J. Distance Educ., vol. 22, no. 2, pp. 1–22, 2008. Available: https://www.ijede.ca/index.php/jde/issue/archive.
  5. J. Biggs and C. Tang, Teaching for Quality Learning at University, 4th ed. Maidenhead, UK: Open University Press, 2011. Available: https://www.google.com/books/edition/_/XhjRBrDAESkC.

Authors
Wang, Y.
Affiliations
Shinawatra University (TH)
Corresponding author
Wang, Yanping
ORCID
Wang, Yanping
Venue
Proceedings of the 2026 International Conference on Artificial Intelligence, Digital Media and Education
Publisher
ACM
Published
2026-05-22
Pages
469-475
DOI
10.1145/3839592.3839666
OpenAlex
W7220844193
Semantic Scholar
6cf44f9634526c5b81743ae2f5d026285024c4ed
Cited by
0
References
11
FWCI
0.00
Type
conference paper
Open access
Licence
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Educational Technology and Pedagogy
  • Social Sciences › Education › Evaluation of Teaching Practices
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
Keywords
classroom interaction, teaching effectiveness, higher education, artificial intelligence, student engagement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-25.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-25.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review