CoChat
Sign Up
Cited by 15Academic integrity and plagiarismSoftware Engineering ResearchSpam and Phishing Detection

Testing of support tools for plagiarism detection

Open access
Foltýnek, Tomáš, Dlabolová, Dita, Anohina-Naumeca, Alla, Razı, Salim, Kravjar, Július, Kamzola, Laima, Guerrero-Dib, Jean, Çelik, Özgür +1 more·2020·International Journal of Educational Technology in Higher Education
DOI 10.1186/s41239-020-00192-4
Open this paper in CoChat — free accountOpen PDF

Abstract

There is a general belief that software must be able to easily do things that humans find difficult. Since finding sources for plagiarism in a text is not an easy task, there is a wide-spread expectation that it must be simple for software to determine if a text is plagiarized or not. Software cannot determine plagiarism, but it can work as a support tool for identifying some text similarity that may constitute plagiarism. But how well do the various systems work? This paper reports on a collaborative test of 15 web-based text-matching systems that can be used when plagiarism is suspected. It was conducted by researchers from seven countries using test material in eight different languages, evaluating the effectiveness of the systems on single-source and multi-source documents. A usability examination was also performed. The sobering results show that although some systems can indeed help identify some plagiarized content, they clearly do not find all plagiarism and at times also identify non-plagiarized material as problematic.

Integrity signals

Registry

Crossref · 10.1186/s41239-020-00192-4

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Text-matching software not only do not find all plagiarism, but furthermore will also mark non-plagiarised content as plagiarism, thus providing false positive results.

    “previous research by Foltýnek et al. (2020) has shown that text-matching software not only do not find all plagiarism, but furthermore will also mark non-plagiarised content as plagiarism, thus providing false positive results.”

    From Testing of detection tools for AI-generated text (2023) · cites this paper as Foltýnek et al. (2020)

  • It is possible to provide evidence of potential misconduct with text matching software.

    “Although text matching software also suffers from false positives and false negatives (Foltýnek et al. 2020), at least it is possible to provide evidence of potential misconduct.”

    From Testing of detection tools for AI-generated text (2023) · cites this paper as Foltýnek et al. 2020

  • Text matching software also suffers from false positives and false negatives.

    “Although text matching software also suffers from false positives and false negatives (Foltýnek et al. 2020), at least it is possible to provide evidence of potential misconduct.”

    From Testing of detection tools for AI-generated text (2023) · cites this paper as Foltýnek et al. 2020

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Foltýnek, T., Dlabolová, D., Anohina-Naumeca, A., Razı, S., Kravjar, J., Kamzola, L., Guerrero-Dib, J., Çelik, Ö., & Weber-Wulff, D.
Affiliations
  • University of Wuppertal (DE)
  • Mendel University in Brno (CZ)
  • Riga Technical University (LV)
  • Çanakkale Onsekiz Mart Üniversitesi (TR)
  • Slovak Centre of Scientific and Technical Information (SK)
  • University of Monterrey (MX)
  • Balıkesir University (TR)
  • HTW Berlin - University of Applied Sciences (DE)
Corresponding author
Foltýnek, Tomáš
ORCID
Venue
International Journal of Educational Technology in Higher Education
Publisher
Springer Science and Business Media LLC
Published
2020-07-27
Volume
17
Issue
1
ISSN
2365-9440 (ISSN-L)
DOI
10.1186/s41239-020-00192-4
OpenAlex
W3006405348
Cited by
15
References
1
Citations by year
2024: 3 · 2023: 5 · 2022: 2 · 2021: 4 · 2020: 1
FWCI
4.13
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Educational Technology in Higher Education (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University) (cc-by)
Funders
  • Università della Calabria
  • Mendelova Univerzita v Brně
  • Universität Konstanz
  • University of Nicosia
Topics
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
Keywords
plagiarism detection, text similarity, web-based tool, usability, cross-lingual evaluation, collaborative assessment, academic integrity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review