CoChat

Rethinking Homework in the Age of Artificial Intelligence

Open access
Ibrahim, Hazem, Asim, Rohail, Zaffar, Fareed, Rahwan, Talal, Zaki, Yasir · 2023 · IEEE Intelligent Systems

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

The evolution of natural language processing techniques has led to the development of advanced conversational tools such as ChatGPT, capable of assisting users with a variety of activities. Media attention has centered on ChatGPT’s potential impact, policy implications, and ethical ramifications, particularly in the context of education. As such tools become more accessible, students across the globe may use them to assist with their homework. However, it is still unclear whether ChatGPT’s performance is advanced enough to pose a serious risk of plagiarism. We fill this gap by evaluating ChatGPT on two introductory and two advanced university-level courses. We find that ChatGPT receives near-perfect grades on the majority of questions in the introductory courses but has not yet reached the level of sophistication required to pass in advanced courses. Moreover, adding a few full stops or typos may fool a machine learning algorithm designed to detect ChatGPT-generated text. These findings suggest that, at least for some courses, current artificial intelligence tools pose a real threat that can no longer be overlooked by educational institutions.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/mis.2023.3255599

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Figures

FIGURE 1 (a) Comparison of the grade distribution of ChatGPT submissions against that of student submissions in four courses. (b) The average grade given to the best ChatGPT answers for each question. (c) The output of a machine learning algorithm designed speci fi cally to quantify the likelihood that a body of text was written by ChatGPT; this algorithm was given three different inputs: 1) the original ChatGPT essays, 2) the ChatGPT essays after inserting up to three full stops per essay, and 3) the ChatGPT essays after inserting up to six typos per essay. The p value is calculated using a paired t test: ** p < 0.01; *** p < 0.001. Intro: Introduction; Q: question.
FIGURE 1 (a) Comparison of the grade distribution of ChatGPT submissions against that of student submissions in four courses. (b) The average grade given to the best ChatGPT answers for each question. (c) The output of a machine learning algorithm designed speci fi cally to quantify the likelihood that a body of text was written by ChatGPT; this algorithm was given three different inputs: 1) the original ChatGPT essays, 2) the ChatGPT essays after inserting up to three full stops per essay, and 3) the ChatGPT essays after inserting up to six typos per essay. The p value is calculated using a paired t test: ** p < 0.01; *** p < 0.001. Intro: Introduction; Q: question.CC-BY · From the PDF
Carnegie Mellon University Software Engineering Institute
Carnegie Mellon University Software Engineering InstituteCC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 3 references

Research works printed but not linked: 8

  1. yang, 2023, New York city schools ban AI chatbot that writes essays and answers prompts
  2. vincent, 2023, The scary truth about AI copyright is nobody knows what will happen next
  3. nisa, 2023, Here’s a list of some of the exams chatGPT has passed so far
  4. face, 0, Gpt-2 output detector demo
  5. habash, 0, ZAEBUC: An annotated Arabic-English bilingual writer corpus, Proc 13th Lang Resour Eval Conf, 79
  6. roose, 2022, An A I -generated picture won an art prize Artists aren’t happy
  7. meckler, 2022, Teachers are on alert for inevitable cheating after release of ChatGPT
  8. 0, ChatGPT Optimizing Language Models for Dialogue

Cited by

See all 73

Metadata

Authors
Ibrahim, H., Asim, R., Zaffar, F., Rahwan, T., & Zaki, Y.
Affiliations
  • New York University Abu Dhabi (AE)
  • Lahore University of Management Sciences (PK)
Venue
IEEE Intelligent Systems
Published
2023-03
Volume
38
Issue
2
Pages
24-27
ISSN
  • 1541-1672 (ISSN-L)
  • 1941-1294
OpenAlex
W4367330207
Cited by
65
References
2
Citations by year
2026: 13 · 2025: 18 · 2024: 23 · 2023: 11
FWCI
15.26
Type
journal article
Language
en
Open access
Locations
IEEE Intelligent Systems (publishedVersion, cc-by)
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Social Sciences › Education › Academic integrity and plagiarism
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
ChatGPT, homework, academic integrity, plagiarism, AI-generated text detection, large language models, university courses, course difficulty, artificial intelligence in education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Rethinking Homework in the Age of Artificial Intelligence. https://app.cochat.ai/papers/10.1109/mis.2023.3255599
The paper
Ibrahim, H., Asim, R., Zaffar, F., Rahwan, T., & Zaki, Y. (2023). Rethinking Homework in the Age of Artificial Intelligence. IEEE Intelligent Systems. https://doi.org/10.1109/mis.2023.3255599

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.