CoChat
Sign Up
Cited by 300Artificial Intelligence in EducationAcademic integrity and plagiarismArtificial Intelligence in Healthcare and Education

Evaluating Academic Answers Generated Using ChatGPT

Open access
Fergus, Suzanne, Botha, Michelle, Ostovar, Mehrnoosh·2023·Journal of Chemical Education
DOI 10.1021/acs.jchemed.3c00087
Open this paper in CoChat — free accountOpen PDF

Abstract

High Resolution Image Download MS PowerPoint Slide The integration of technology in education has become ever more prioritized since the COVID-19 pandemic. Chat Generative Pre-Trained Transformer (ChatGPT) is an artificial intelligence technology that generates conversational interactions to user prompts. The trained model can answer follow-up questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests. The functionality of ChatGPT in answering chemistry assessment questions requires investigation to ascertain its potential impact on learning and assessment. Two chemistry-focused modules in year 1 and year 2 of a pharmaceutical science program are used to study and evaluate ChatGPT-generated responses in relation to the end-of-year exam assessments. For questions that focused on knowledge and understanding with “describe” and “discuss” verbs, the ChatGPT generated responses. For questions that focused on application of knowledge and interpretation with nontext information, the ChatGPT technology reached a limitation. A further analysis of the quality of responses is reported in this study. ChatGPT is not considered a high-risk technology tool in relation to cheating. Similar to the COVID-19 disruption, ChatGPT is expected to provide a catalyst for educational discussions on academic integrity and assessment design.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1021/acs.jchemed.3c00087

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Various assignment types, including essays and computer coding, are impacted differently by GenAI.

    “Various assignment types, including essays and computer coding, are impacted differently by GenAI (Fergus et al., 2023; Greiner et al., 2023; Guo & Wang, 2023; Naidu & Sevnarayan, 2023; Ouh et al., 2023; Smolansky et al., 2023).”

    From A Scoping Review on How Generative Artificial Intelligence Transforms Assessment in Higher Education (2024) · cites this paper as Fergus et al., 2023

  • Concerns have been raised that ChatGPT may disseminate false information and references, reinforce bias, compromise academic integrity, and cause potential decline in students' skills.

    “These include the dissemination of false information and references [12, 23, 61, 85], biased reinforcement [47, 50], compromised academic integrity [18, 40, 66, 74], and the potential decline in students' skills [43, 61, 64, 74].”

    From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [23]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Fergus, S., Botha, M., & Ostovar, M.
Affiliations
University of Hertfordshire (GB)
Corresponding author
Fergus, Suzanne
ORCID
Venue
Journal of Chemical Education
Publisher
American Chemical Society (ACS)
Published
2023-03-31
Volume
100
Issue
4
Pages
1672-1675
ISSN
  • 0021-9584 (ISSN-L)
  • 1938-1328
DOI
10.1021/acs.jchemed.3c00087
OpenAlex
W4362471626
Cited by
300
References
13
Citations by year
2026: 52 · 2025: 68 · 2024: 117 · 2023: 63
FWCI
70.41
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Chemical Education (publishedVersion, cc-by)
  • University of Hertfordshire Research Archive (University of Hertfordshire) (acceptedVersion)
  • University of Hertfordshire Research Archive (University of Hertfordshire) (submittedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
ChatGPT, large language models, chemistry assessment, academic integrity, cheating, assessment design, AI-generated responses, COVID-19 pandemic disruption

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review