CoChat

Using Machine Learning to Detect ‘Multiple-Account’ Cheating and Analyze the Influence of Student and Problem Features

Open access
Ruiperez-Valiente, Jose A., Munoz-Merino, Pedro J., Alexandron, Giora, Pritchard, David E. · 2019 · IEEE Transactions on Learning Technologies
Cited by 60

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

One of the reported methods of cheating in online environments in the literature is CAMEO (Copying Answers using Multiple Existences Online), where harvesting accounts are used to obtain correct answers that are later submitted in the master account which gives the student credit to obtain a certificate. In previous research, we developed an algorithm to identify and label submissions that were cheated using the CAMEO method; this algorithm relied on the IP of the submissions. In this study, we use this tagged sample of submissions to i) compare the influence of student and problems characteristics on CAMEO and ii) build a random forest classifier that detects submissions as CAMEO without relying on IP, achieving sensitivity and specificity levels of 0.966 and 0.996, respectively. Finally, we analyze the importance of the different features of the model finding that student features are the most important variables towards the correct classification of CAMEO submissions, concluding also that student features have more influence on CAMEO than problem features.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tlt.2017.2784420

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 25

  1. 10.1016/j.compedu.2017.01.015 · DOI
  2. baker, 2008, Why students engage in “gaming the system, J Interact Learn Res, 19
  3. 10.1103/PhysRevSTPER.6.010104 · DOI · Paper page
  4. 10.1007/s11257-011-9106-8 · DOI · Paper page
  5. arroyo, 2005, Inferring learning and attitudes from a Bayesian network of log file data, Proc 12th Int Conf Artif Intell, 33
  6. baker, 2009, Educational software features that encourage and discourage gaming the system, Proc 14th Int Conf Artificial Intell, 475
  7. 10.1007/978-3-642-13388-6_23 · DOI
  8. 10.1007/s11257-010-9086-0 · DOI
  9. bogle, 2000, Effect of perspective, type of student, and gender on the attribution of cheating, Proc Oklahoma Academic Sci Oklahoma City, 80, 91
  10. 10.32614/RJ-2015-018 · DOI · Paper page
  11. galbraith, 2010, Understanding incivility in online teaching, J Adult Edu, 39
  12. louppe, 2013, Understanding variable importances in forests of randomized trees, Proc Int Conf Adv Neural Inform Process Syst, 431
  13. 10.3200/JECE.39.2.116-125 · DOI · Paper page
  14. yousef, 2014, MOOCs: A review of the state-of-the-art, Proc 6th Int Conf Comput Supported Edu, 9
  15. 10.1145/3041021.3054161 · DOI
  16. siemens, 2013, Massive open online courses: Innovation in education, Open Educational Resources Innovation Research and Practice, 5, 5
  17. 10.1145/2876034.2876037 · DOI
  18. 10.1016/j.compedu.2016.04.008 · DOI
  19. alexandron, 2016, Detecting cheaters in MOOCs using item response theory and learning analytics, Proc 6th Workshop Personalization Approaches Learn Environ
  20. lambert, 2003, Collegiate academic dishonesty revisited: What have they done, how often have they done it, who does it, and why did they do it, J Electronic Soc, 7, 1
  21. balbuena, 2015, Prevalence, motives, and views of academic dishonesty in higher education, Pabico Asia Pacific Journal of Multidisciplinary Research, 3, 69
  22. 10.1080/10508422.2015.1051661 · DOI
  23. 10.1016/j.compedu.2012.06.001 · DOI
  24. 10.1016/j.compedu.2014.12.016 · DOI
  25. 10.1080/10508422.2015.1009630 · DOI

Cited by

See all 60

Metadata

Authors
Ruiperez-Valiente, J. A., Munoz-Merino, P. J., Alexandron, G., & Pritchard, D. E.
Venue
IEEE Transactions on Learning Technologies
Published
2019-01-01
Volume
12
Issue
1
Pages
112-122
ISSN
  • 1939-1382 (ISSN-L)
  • 2372-0050
OpenAlex
W2780493198
Cited by
60
References
3
FWCI
10.37
Type
journal article
Language
en
Open access
green
Funders
  • Madrid Regional Government (S2013/ICE-2715)
  • Spanish Ministry of Economy and Competitiveness (TIN2014-53199-C3-1-R)
  • European Erasmus+ projects MOOC Maker) (561533-EPP-1-2015-1-ES-EPPKA2-CBHE-JP)
  • SHEILA (562080-EPP-1-2015-BE-EPPKA3-PI-FORWARD)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.