Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses
Open accessThe DOI matches its Crossref record; 70 of the 71 references it cites were found in a registry or an index; 1 of the 22 sources its claims cite were checked for support.
Abstract
Several tools and resources have been developed in the past years to enhance the teaching and learning process. Most of them are focused on the process itself, but few focus on the assessment process to detect at-risk learners for later acting through feedback to support them to succeed and pass the course. This research paper presents a case study using an adaptive system called Learning Intelligent System (LIS). The system includes an Early Warning System and tested in a fully online university to increase learners’ performance, reduce dropout, and ensure proper feedback to guide learners. LIS also aims to help teachers to detect critical cases to act on time with learners. The system has been tested in two first-year courses in the fully online BSc of Economics and Business at the Universitat Oberta de Catalunya. A total of 552 learners were participating in the case study. On the one hand, results show that performance is better than in previous semesters when using it. On the other hand, results show that learners' perception of effectiveness is higher, and learners are willing to continue using the system in the following semesters because it becomes beneficial for them.
Overview
CoChat Summary
TL;DR
This case study tested LIS, an adaptive early-warning system, in two fully online first-year Economics and Business courses at Universitat Oberta de Catalunya with 552 consenting learners. It reports better final marks and lower dropout for LIS users and positive learner perceptions of the system’s effectiveness and usefulness.
Key findings
- Dropout after ALA4 was lower for LIS-signed learners than for not-signed and previous-semester groups: Introduction to enterprise 14.20% vs 32.79% vs 29.24%; Markets and behavior 9.52% vs 31.90% vs 18.2%.
- Final marks were significantly better for the signed LIS group: Introduction to enterprise signed vs not signed p=0.0048 and signed vs previous semester p=2.6e-16; Markets and behavior signed vs not signed p=2e-6 and signed vs previous semester p=2.2e-6.
- The GAR model’s global accuracy was >78% at ALA1 and >90% at ALA5 in both courses; TNR was >89%, and TPR exceeded 70% from ALA2. At-risk learners identified ranged from 6.80% to 15.90% in Introduction to enterprise and 9.60% to 14.10% in Markets and behavior.
- Among 205 post-questionnaire respondents, 75.12% found the dashboard useful, 68.29% agreed or strongly agreed to future LIS use, and more than 60% rated feedback messages, dashboard, and LIS globally as “Like somewhat” or “Like a great deal.”
Methods
- Mixed action research plus design-and-creation approach; pilot in two first-year 6 ECTS courses, Markets and behavior and Introduction to enterprise, in the fully online BSc of Economics and Business at Universitat Oberta de Catalunya during the February–June 2020 semester.
- 552 learners consented to participate (44.52% of the total); LIS used a Gradual At-Risk model trained on historical anonymized data with Naive Bayes, Decision Tree, K-Nearest Neighbors, and Support Vector Machine, assigning Green/Yellow/Red/NS1/NS2 risk levels and sending dashboards and emails.
- Quantitative analysis used the unpaired two-sample Wilcoxon test for final marks, descriptive analysis of dropout and ALA performance, and MARS multivariate regression; the post-questionnaire was administered via Qualtrics with 205 responses on Likert scales.
Limitations
- Stated: self-selection bias is induced by the Ethical Committee’s consent requirement, since learners who participate in pilots tend to have better performance and a predisposition to test innovative tools.
- Stated: mortality bias may affect the qualitative analysis because not all pilot participants answered the post-questionnaire, and failed or dropped-out learners likely did not answer; anonymity prevents checking this.
- Stated: the GAR model becomes invalid if a course’s continuous assessment changes drastically in number or type of activities, requiring contingency models; evident from the reported age distribution, post-questionnaire respondents were skewed younger (31.71% aged 21–25), which may limit generalizability of perception findings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
1 Valid · 1 Not valid · 13 Not checked · 96 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“there are different situations, and learners from each group will have different needs (Grau-Valldosera and Minguillón 2014).”
§ Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses > Study procedure and instruments
What the sources say
- Partly supportedcites Grau-Valldosera and Minguillón 2014 sourceExperimental — accuracy not yet measured
Our copy supports the general idea that different groups of non-continuing learners sit in different situations requiring different institutional responses: the Conclusions (pp. 304-305) distinguish early from midterm dropout as 'probably closer to the concept of attrition and caused by different reasons', call for corrective actions targeted at different sub-groups (e.g. first-semester leavers needing mentoring), and say a finer-tuned analysis helps institutions allocate scarce resources. However, the specific contrast the citing sentence draws — learners who FAIL versus learners who DROP OUT having different needs — is not made by Grau-Valldosera and Minguillón; the only place their paper separates failing students ('non-successful completers', 'academic withdrawal … students who fail two courses') from withdrawers is Table 1, which is a compilation of other authors' definitions extracted from Lee & Choi (2011), and it concerns definitional heterogeneity, not learner needs. So part of what is attributed (the fail-vs-dropout grouping and its consequences for needs) is not in the source. (LLM, conf=0.65)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“learners are willing to continue using the system in the following semesters because it becomes beneficial for them.”
§ Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the other hand, results show that learners' perception of effectiveness is higher”
§ Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the one hand, results show that performance is better than in previous semesters when using it.”
§ Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Only 27.80% of learners' answers are located between strongly disagree, disagree, and somewhat disagree.”
§ Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses > Related to RQ2. Do learners consider that the Learning Intelligent System is effective and useful?
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
56 more claims
Create a free account to see all 111 claims and to verify the 21 references still open.
- Probably not a substantial claim. 28
- It may state two separate claims as one. 28
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, to many educators, it is their first experience to use AI technologies in an online learning environment, and they may not have adequate knowledge and skills to manipulate these AI applications (Guerrero-Roldan et al., 2021).”
From Teachers’ AI digital competencies and twenty-first century skills in the post-pandemic world (2023) · doi:10.1007/s11423-023-10203-6 · cites this paper as Guerrero-Roldan et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses > Related to RQ 1. Is learners' performance increasing when using the Learning Intelligent System?
“Green risk level correctly identifies learners who pass in both courses in all activities with an accuracy higher than 90%.”
§ Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses > Conclusions, limitations, and future research
“| G | 152 88.2 | 11.8 | 159 87.4 | 12.6 | 153 90.9 | 9.1 | 154 92.2 | 7.8 | 148 95.3 | 4.7 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses > Related to RQ 1. Is learners' performance increasing when using the Learning Intelligent System?
“in both courses, the TNR value is higher than 89%”
§ Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses > Conclusions, limitations, and future research
“| ALA 2 | 82.41 | 86.56 | 74.10 | 74.39 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses > Related to RQ2. Do learners consider that the Learning Intelligent System is effective and useful?
“All questions were appraised with the largest scale options (Like somewhat, Like a great deal) with more than 60% of the responses.”
§ Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses > Conclusions, limitations, and future research
“| LIS system globally | 6.34 | 7.80 | 26.83 | 40.49 | 18.54 |”
Citations
Cites
Multivariate Adaptive Regression Splines
Friedman, J. H. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347963 · Cited by 8,000 · cited 2×
“Here, we show the result model based on the multivariate regression splines (MARS) on the Markets and behavior course.”
MethodConditions Under Which Assessment Supports Students’ Learning
Gibbs, G., & Simpson, C. (2005) · Research Repository (University of Gloucestershire) · Cited by 1,502
Researching Information Systems and Computing
Oates, B., Griffiths, M., & McLean, R. (2022) · SAGE Publications Ltd eBooks · doi:10.4135/9781036231330 · Cited by 1,191
Learning analytics and educational data mining: towards communication and collaboration
Siemens, G., & Baker, R. S. J. D. (2012) · Proceedings of the 2nd International Conference on Learning Analytics and Knowledge · doi:10.1145/2330601.2330661 · Cited by 1,075 · cited 1×
“Learning analytics and educational data mining research areas explore how collected and analyzed data from digital systems can enhance the teaching and learning process (Siemens and Baker 2012).”
BackgroundMining LMS data to develop an “early warning system” for educators: A proof of concept
Macfadyen, L. P., & Dawson, S. (2010) · Computers & Education · doi:10.1016/j.compedu.2009.09.008 · Cited by 1,045
Course signals at Purdue: using learning analytics to increase student success
Arnold, K. E., & Pistilli, M. D. (2012) · Proceedings of the 2nd International Conference on Learning Analytics and Knowledge · doi:10.1145/2330601.2330666 · Cited by 987 · cited 3×
“For example, Arnold and Pistilli (2012) detected a strong increase in satisfactory grades, and a decrease in unsatisfactory grades and dropout.”
BackgroundSun, J. C., & Rueda, R. (2012) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2010.01157.x · Cited by 785
Mousavinasab, E., Zarifsanaiey, N., R. Niakan Kalhori, S., Rakhshan, M., Keikha, L., & Ghazi Saeedi, M. (2021) · Interactive Learning Environments · doi:10.1080/10494820.2018.1558257 · Cited by 605
Blended learning in higher education: Trends and capabilities
Castro, R. (2019) · Education and Information Technologies · doi:10.1007/s10639-019-09886-3 · Cited by 552
Intelligent tutoring systems : lessons learned
Psotka, J., Massey, L., & Mutter, S. A. (1988) · Lawrence Erlbaum Associates eBooks · Cited by 457
Research works printed but not linked: 2
- Cem, F. (2008). A multi-agent adaptive learning system for distance education (Middle East Technical University). https://www.semanticscholar.org/paper/A-MULTI-AGENT-ADAPTIVE-LEARNING-SYSTEM-FOR-DISTANCE-Cem/bef6e4c4541d2aca91d0de0f1c9cf09def02eedc.
- Melton, R. F. (2002). Planning and developing open and distance learning: a quality assurance approach. Planning and developing open and distance learning: a quality assurance approach. London: Routledge Falmer.
Cited by
Teachers’ AI digital competencies and twenty-first century skills in the post-pandemic world
(2023) · doi:10.1007/s11423-023-10203-6 · Cited by 734
“…need to equip themselves with AI-related technological skills to facilitate students’ knowledge acquisition and expression (Healy & Blade, 2020), and interact with learners using AI technologies (e.g., chatbots, automatic feedback) (Guerrero-Roldan et al., 2021; Whitelock-Wainwright et al., 2021).”
BackgroundMethodInfluentialEnhancing Student Engagement: Harnessing “AIED”’s Power in Hybrid Education—A Review Analysis
Almusaed, A., Almssad, A., Yitmen, I., & Homod, R. Z. (2023) · Education Sciences · doi:10.3390/educsci13070632 · Cited by 242
“This lets them obtain immediate feedback on their exercises and assignments [7].”
BackgroundAn early warning system to identify and intervene online dropout learners
Bañeres, D., Rodríguez, M. E., Guerrero‐Roldán, A., & Cortadas, P. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-022-00371-5 · Cited by 153
“Some works are not distinguishing between them (Boudjehem & Lafifi, 2021; Guerrero-Roldán et al., 2021), and the interventions may not be correctly aligned with the actual risk.”
BackgroundMethodInfluentialSelf-citationPelima, L. R., Sukmana, Y., & Rosmansyah, Y. (2024) · IEEE Access · doi:10.1109/access.2024.3361479 · Cited by 150
“The financial reliance status refers to the extent to which a student relies on their parents’ financial assistance [13], [18], [44], [45], [49], [57], [63], [71], [79].”
BackgroundInfluentialWu, T.-T., Lee, H.-Y., Wang, W.-S., Lin, C.-J., & Huang, Y.-M. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00422-5 · Cited by 82
Metadata
- Authors
- Guerrero-Roldán, A.-E., Rodríguez-González, M. E., Bañeres, D., Elasri-Ejjaberi, A., & Cortadas, P.
- Affiliations
- Universitat Oberta de Catalunya (ES)
- Venue
- International Journal of Educational Technology in Higher Education
- Published
- 2021-07-02
- Volume
- 18
- Issue
- 1
- ISSN
- 2365-9440 (ISSN-L)
- OpenAlex
- W3176899830
- Cited by
- 50
- References
- 71
- Citations by year
- 2026: 7 · 2025: 11 · 2024: 13 · 2023: 14 · 2022: 4 · 2021: 1
- FWCI
- 6.77
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- International Journal of Educational Technology in Higher Education (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- O2 - Repositori Institucional (Universitat Oberta de Catalunya) (submittedVersion, cc-by)
- Funders
- Universitat Oberta de Catalunya (2018NG001)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Edcuational Technology Systems
- Keywords
- adaptive intelligent systems, early warning systems, online learning, school dropout prevention, at-risk students, learner performance, feedback, first-year students, economics and business education, learner perceptions, Universitat Oberta de Catalunya
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses. https://app.cochat.ai/papers/10.1186/s41239-021-00271-0
- The paper
- Guerrero-Roldán, A.-E., Rodríguez-González, M. E., Bañeres, D., Elasri-Ejjaberi, A., & Cortadas, P. (2021). Experiences in the use of an adaptive intelligent system to enhance online learners' performance: a case study in Economics and Business courses. International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-021-00271-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.