Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.

DeepLMS: a deep learning predictive model for supporting online learning in the Covid-19 era

Open access

Abstract

Coronavirus (Covid-19) pandemic has imposed a complete shut-down of face-to-face teaching to universities and schools, forcing a crash course for online learning plans and technology for students and faculty. In the midst of this unprecedented crisis, video conferencing platforms (e.g., Zoom, WebEx, MS Teams) and learning management systems (LMSs), like Moodle, Blackboard and Google Classroom, are being adopted and heavily used as online learning environments (OLEs). However, as such media solely provide the platform for e-interaction, effective methods that can be used to predict the learner’s behavior in the OLEs, which should be available as supportive tools to educators and metacognitive triggers to learners. Here we show, for the first time, that Deep Learning techniques can be used to handle LMS users’ interaction data and form a novel predictive model, namely DeepLMS, that can forecast the quality of interaction (QoI) with LMS. Using Long Short-Term Memory (LSTM) networks, DeepLMS results in average testing Root Mean Square Error (RMSE)$$<0.009$$<0.009, and average correlation coefficient between ground truth and predicted QoI values$$r\ge 0.97$$r≥0.97$$(p<0.05)$$(p<0.05), when tested on QoI data from one database pre- and two ones during-Covid-19 pandemic. DeepLMS personalized QoI forecasting scaffolds user’s online learning engagement and provides educators with an evaluation path, additionally to the content-related assessment, enriching the overall view on the learners’ motivation and participation in the learning process.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41598-020-76740-9

Cites

76 of 76 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 The DeepLMS-based QoI prediction concept. A schematic representation of the proposed DeepLMS functionality, with the LMS Moodle user’s interaction metrics (M1,…,M110; see Supplementary Table S1) categorized into 14 input parameters (C1,…,C14; see Supplementary Table S1) fed to the FuzzyQoI model42, outputting the estimated QoI(k) at instance k. The latter is then inputted to the trained LMST network (see “Methods” section) to predict the QoI^(k+1) at instance (k+1). Both QoI(k) and QoI^(k+1) are compared and their difference (dQoI(k)) is used to inform the user’s feedback path.
Figure 1 The DeepLMS-based QoI prediction concept. A schematic representation of the proposed DeepLMS functionality, with the LMS Moodle user’s interaction metrics (M1,…,M110; see Supplementary Table S1) categorized into 14 input parameters (C1,…,C14; see Supplementary Table S1) fed to the FuzzyQoI model42, outputting the estimated QoI(k) at instance k. The latter is then inputted to the trained LMST network (see “Methods” section) to predict the QoI^(k+1) at instance (k+1). Both QoI(k) and QoI^(k+1) are compared and their difference (dQoI(k)) is used to inform the user’s feedback path.CC BY · PubMed Central
Figure 2 Predictive performance of the DeepLMS on QoI time series from DB1 Professors. The left column shows the QoI data from DB1 P#2, P#33, P#35, P#60, P#65, and P#70, used for training (from day 1 until day 323 where the vertical solid line lies) and for testing (day 324 until day 358), whereas the right column zooms into the testing QoI data (blue solid line) and the DeepLMS predicted QoI (red dashed line). Moreover, the estimated correlation coefficient r between the testing and the estimated QoI data (see “Methods” section) for each case is also superimposed in the right column plots.
Figure 2 Predictive performance of the DeepLMS on QoI time series from DB1 Professors. The left column shows the QoI data from DB1 P#2, P#33, P#35, P#60, P#65, and P#70, used for training (from day 1 until day 323 where the vertical solid line lies) and for testing (day 324 until day 358), whereas the right column zooms into the testing QoI data (blue solid line) and the DeepLMS predicted QoI (red dashed line). Moreover, the estimated correlation coefficient r between the testing and the estimated QoI data (see “Methods” section) for each case is also superimposed in the right column plots.CC BY · PubMed Central
Figure 3 Predictive performance of the DeepLMS on QoI time series from DB1 Students. The left column shows the QoI data from DB1 S#55, S#60, S#155, S#310, S#612, and S#775, used for training (from day 1 until day 323 where the vertical solid line lies) and for testing (day 324 until day 358), whereas the right column zooms into the testing QoI data (blue solid line) and the DeepLMS predicted QoI (red dashed line). Moreover, the estimated correlation coefficient r between the testing and the estimated QoI data (see “Methods” section) for each case is also superimposed in the right column plots.
Figure 3 Predictive performance of the DeepLMS on QoI time series from DB1 Students. The left column shows the QoI data from DB1 S#55, S#60, S#155, S#310, S#612, and S#775, used for training (from day 1 until day 323 where the vertical solid line lies) and for testing (day 324 until day 358), whereas the right column zooms into the testing QoI data (blue solid line) and the DeepLMS predicted QoI (red dashed line). Moreover, the estimated correlation coefficient r between the testing and the estimated QoI data (see “Methods” section) for each case is also superimposed in the right column plots.CC BY · PubMed Central
Figure 4 Distribution of the DeepLMS predictive performance indices across users’ groups of DB1. (a) Box-plot of the distribution of RMSE between the testing and the estimated QoI data for the case of DB1 Professors, (b) box-plot of the distribution of the correlation coefficient r between the testing and the estimated QoI data for the case of DB1 Professors, (c) box-plot of the distribution of the correlation coefficient rd between the derivative of the testing and the derivative of the estimated QoI data for the case of DB1 Professors; (d-f) same as (a–c), respectively, yet for the case of DB1 Students. Each box-plot visualises the interquartile range (height of rectangle), spanning the first (bottom) to the third quartile (top), the median value (horizontal red line inside the rectangle), the minimum and maximum values (ends of “whiskers” below and above the box, respectively) still within the interquartile range, and outlier values (individual red crosses below and above “whiskers”). Additional DeepLMS predictive performance indices are tabulated in Table 1.
Figure 4 Distribution of the DeepLMS predictive performance indices across users’ groups of DB1. (a) Box-plot of the distribution of RMSE between the testing and the estimated QoI data for the case of DB1 Professors, (b) box-plot of the distribution of the correlation coefficient r between the testing and the estimated QoI data for the case of DB1 Professors, (c) box-plot of the distribution of the correlation coefficient rd between the derivative of the testing and the derivative of the estimated QoI data for the case of DB1 Professors; (d-f) same as (a–c), respectively, yet for the case of DB1 Students. Each box-plot visualises the interquartile range (height of rectangle), spanning the first (bottom) to the third quartile (top), the median value (horizontal red line inside the rectangle), the minimum and maximum values (ends of “whiskers” below and above the box, respectively) still within the interquartile range, and outlier values (individual red crosses below and above “whiskers”). Additional DeepLMS predictive performance indices are tabulated in Table 1.CC BY · PubMed Central
Figure 5 Predictive performance of the DeepLMS on QoI time series from DB2 Professors and Students. The left column-top panel shows the QoI data from the three DB2 Professors, i.e., P#1, P#2, and P#3, used for training (from day 1 until day 68 where the vertical solid line lies) and for testing (day 69 until day 76), whereas the left column-bottom panel shows the QoI data from excerpts of DB2 Students, i.e., S#25, S#39, S#58, S#158, S#171, and S#172, for the same training and testing periods. The right column (both panels) zooms into the testing QoI data (blue solid line) and the DeepLMS predicted QoI (red dashed line), including also the estimated correlation coefficient r between the testing and the estimated QoI data for each case.
Figure 5 Predictive performance of the DeepLMS on QoI time series from DB2 Professors and Students. The left column-top panel shows the QoI data from the three DB2 Professors, i.e., P#1, P#2, and P#3, used for training (from day 1 until day 68 where the vertical solid line lies) and for testing (day 69 until day 76), whereas the left column-bottom panel shows the QoI data from excerpts of DB2 Students, i.e., S#25, S#39, S#58, S#158, S#171, and S#172, for the same training and testing periods. The right column (both panels) zooms into the testing QoI data (blue solid line) and the DeepLMS predicted QoI (red dashed line), including also the estimated correlation coefficient r between the testing and the estimated QoI data for each case.CC BY · PubMed Central
Figure 6 Distribution of the DeepLMS predictive performance indices across users’ groups of DB2. (a) Box-plot of the distribution of RMSE between the testing and the estimated QoI data for the case of DB2 Professors, (b) box-plot of the distribution of the correlation coefficient r between the testing and the estimated QoI data for the case of DB2 Professors, (c) box-plot of the distribution of the correlation coefficient rd between the derivative of the testing and the derivative of the estimated QoI data for the case of DB2 Professors; (d–f) same as (a–c), respectively, yet for the case of DB2 Students. Additional DeepLMS predictive performance indices are tabulated in Table 1.
Figure 6 Distribution of the DeepLMS predictive performance indices across users’ groups of DB2. (a) Box-plot of the distribution of RMSE between the testing and the estimated QoI data for the case of DB2 Professors, (b) box-plot of the distribution of the correlation coefficient r between the testing and the estimated QoI data for the case of DB2 Professors, (c) box-plot of the distribution of the correlation coefficient rd between the derivative of the testing and the derivative of the estimated QoI data for the case of DB2 Professors; (d–f) same as (a–c), respectively, yet for the case of DB2 Students. Additional DeepLMS predictive performance indices are tabulated in Table 1.CC BY · PubMed Central
Figure 7 Predictive performance of the DeepLMS on QoI time series from DB3 Professor and Students. The left column-top panel shows the QoI data from the one DB3 Professor, i.e., P#1, used for training (from day 1 until day 163 where the vertical solid line lies) and for testing (day 164 until day 181), whereas the left column-bottom panel shows the QoI data from excerpts of DB3 Students, i.e., S#13, S#18, S#23, S#27, S#29, and S#40, for the same training and testing periods. The right column (both panels) zooms into the testing QoI data (blue solid line) and the DeepLMS predicted QoI (red dashed line), including also the estimated correlation coefficient r between the testing and the estimated QoI data for each case.
Figure 7 Predictive performance of the DeepLMS on QoI time series from DB3 Professor and Students. The left column-top panel shows the QoI data from the one DB3 Professor, i.e., P#1, used for training (from day 1 until day 163 where the vertical solid line lies) and for testing (day 164 until day 181), whereas the left column-bottom panel shows the QoI data from excerpts of DB3 Students, i.e., S#13, S#18, S#23, S#27, S#29, and S#40, for the same training and testing periods. The right column (both panels) zooms into the testing QoI data (blue solid line) and the DeepLMS predicted QoI (red dashed line), including also the estimated correlation coefficient r between the testing and the estimated QoI data for each case.CC BY · PubMed Central
Figure 8 Distribution of the DeepLMS predictive performance indices across the DB3 Students. (a) Box-plot of the distribution of RMSE between the testing and the estimated QoI data for the case of DB3 Students, (b) box-plot of the distribution of the correlation coefficient r between the testing and the estimated QoI data for the case of DB3 Students, (c) box-plot of the distribution of the correlation coefficient rd between the derivative of the testing and the derivative of the estimated QoI data for the case of DB3 Students. Additional DeepLMS predictive performance indices are tabulated in Table 1.
Figure 8 Distribution of the DeepLMS predictive performance indices across the DB3 Students. (a) Box-plot of the distribution of RMSE between the testing and the estimated QoI data for the case of DB3 Students, (b) box-plot of the distribution of the correlation coefficient r between the testing and the estimated QoI data for the case of DB3 Students, (c) box-plot of the distribution of the correlation coefficient rd between the derivative of the testing and the derivative of the estimated QoI data for the case of DB3 Students. Additional DeepLMS predictive performance indices are tabulated in Table 1.CC BY · PubMed Central
Figure 9 Overview of an LSTM neural processing unit. Structural characteristics of an LSTM unit and its sequence across time. xt is the input data, ht is the hidden state, it, ot, and ft are gates controlling the flow of information, and Ct is the cell state. The x and + operators represent the element-wise product (∘) and summation, respectively, whereas σ denotes the sigmoid function of σ(x)=(1+e^(-x))⁻¹.
Figure 9 Overview of an LSTM neural processing unit. Structural characteristics of an LSTM unit and its sequence across time. xt is the input data, ht is the hidden state, it, ot, and ft are gates controlling the flow of information, and Ct is the cell state. The x and + operators represent the element-wise product (∘) and summation, respectively, whereas σ denotes the sigmoid function of σ(x)=(1+e^(-x))⁻¹.CC BY · PubMed Central
Figure 10 The estimated training RMSE across iterations. An example of the convergence of the estimated training RMSE to values <0.001 across the 300 iterations, when training the proposed LSTM network with data from DB1. The light and thick lines denote the actual and the smoothed values of the estimated training RMSE, respectively.
Figure 10 The estimated training RMSE across iterations. An example of the convergence of the estimated training RMSE to values <0.001 across the 300 iterations, when training the proposed LSTM network with data from DB1. The light and thick lines denote the actual and the smoothed values of the estimated training RMSE, respectively.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 76 references

Research works printed but not linked: 5

  1. Ponce, O. A., Gómez, J. & Pagán, N. Current scientific research in the humanities and social sciences: central issues in educational research. Eur. J. Sci. Theol. 15, 81–95 (2019).
  2. Chen, N.-S. & Lin, K.-M. Factors affecting e-learning for achievement. In IEEE International Conference on Advanced Learning Technologies, Kazan, Russia 200–205 (2002).
  3. Kickul, J. & Kickul, G. New pathways in e-learning: the role of student proactivity and technology utilization. In 45rd Annual Meeting of the Midwest Academy of Management Conference, Indiana, USA (2002).
  4. Laptev, N., Yosinski, J., Li, L. E. & Smyl, S. Time-series extreme event forecasting with neural networks at uber. Int. Conf. Mach. Learn. 34, 1–5 (2017).
  5. student models for interventions. Mao, Y. Deep learning vs. Bayesian knowledge tracing. J. Educ. Data Min. 10, 28–54 (2018).

Cited by

See all 115

Metadata

Authors
Dias, S. B., Hadjileontiadou, S. J., Diniz, J., & Hadjileontiadis, L. J.
Affiliations
  • University of Lisbon (PT)
  • Democritus University of Thrace (GR)
  • Khalifa University of Science and Technology (AE)
Corresponding author
Hadjileontiadis, Leontios J.
Venue
Scientific Reports
Published
2020-11-16
Volume
10
Issue
1
Pages
19888-19888
ISSN
2045-2322 (ISSN-L)
OpenAlex
W3103082384
Cited by
115
References
76
Citations by year
2026: 11 · 2025: 12 · 2024: 20 · 2023: 24 · 2022: 26 · 2021: 20 · 2020: 2
FWCI
16.54
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Informationszentrum Lebenswissenschaften (ZB MED) (submittedVersion, cc-by)
Funders
  • Universidade de Lisboa
  • Research Committee, Aristotle University of Thessaloniki
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Social Sciences › Education › E-Learning and COVID-19
Keywords
online learning, virtual learning environments, interaction quality, COVID-19 pandemic, deep learning, long short-term memory networks, student engagement, student motivation
MeSH terms
Deep Learning, Teleworking, COVID-19, Adolescent, Adult, Computer-Assisted Instruction, Education, Professional, Humans, Middle Aged, Software, Universities

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: DeepLMS: a deep learning predictive model for supporting online learning in the Covid-19 era. https://app.cochat.ai/papers/10.1038/s41598-020-76740-9
The paper
Dias, S. B., Hadjileontiadou, S. J., Diniz, J., & Hadjileontiadis, L. J. (2020). DeepLMS: a deep learning predictive model for supporting online learning in the Covid-19 era. Scientific Reports. https://doi.org/10.1038/s41598-020-76740-9

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.