CoChat
Sign Up
Cited by 106Artificial Intelligence in EducationArtificial Intelligence in Healthcare and EducationArtificial Intelligence Applications

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.

Performance of ChatGPT on the US fundamentals of engineering exam: Comprehensive assessment of proficiency and potential implications for professional environmental engineering practice

Open access
Pursnani, Vinay, Sermet, Yusuf, Kurt, Musa, Demir, Ibrahim·2023·Computers and Education: Artificial Intelligence
DOI 10.1016/j.caeai.2023.100183
Open this paper in CoChat — free account

Abstract

In recent years, advancements in artificial intelligence (AI) have led to the development of large language models like GPT-4, demonstrating potential applications in various fields, including education. This study investigates the feasibility and effectiveness of using ChatGPT, a GPT-4 based model, in achieving satisfactory performance on the Fundamentals of Engineering (FE) Environmental Exam. This study further shows a significant improvement in the model's accuracy when answering FE exam questions through noninvasive prompt modifications, substantiating the utility of prompt modification as a viable approach to enhance AI performance in educational contexts. Furthermore, the findings reflect remarkable improvements in mathematical capabilities across successive iterations of ChatGPT models, showcasing their potential in solving complex engineering problems. Our paper also explores future research directions, emphasizing the importance of addressing AI challenges in education, enhancing accessibility and inclusion for diverse student populations, and developing AI-resistant exam questions to maintain examination integrity. By evaluating the performance of ChatGPT in the context of the FE Environmental Exam, this study contributes valuable insights into the potential applications and limitations of large language models in educational settings. As AI continues to evolve, these findings offer a foundation for further research into the responsible and effective integration of AI models across various disciplines, ultimately optimizing the learning experience and improving student outcomes.

Overview

Integrity signals

Registry

Crossref · 10.1016/j.caeai.2023.100183

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 33 references

Research works printed but not linked: 2

  1. Gilson, 2023, How does CHATGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment, JMIR Medical Education, 9
  2. Hwang, 2023, Editorial position paper: Exploring the potential of generative artificial intelligence in education: Applications, challenges, and future research directions, Educational Technology & Society, 26

Cited by

Sorted by citations

See all 106

Related

Metadata

Authors
Pursnani, V., Sermet, Y., Kurt, M., & Demir, I.
Affiliations
University of Iowa (US)
Corresponding author
Sermet, Yusuf
ORCID
Venue
Computers and Education: Artificial Intelligence
Publisher
Elsevier BV
Published
2023
Volume
5
Pages
100183
ISSN
2666-920X (ISSN-L)
DOI
10.1016/j.caeai.2023.100183
OpenAlex
W4388490623
Cited by
106
References
33
Citations by year
2026: 13 · 2025: 34 · 2024: 50 · 2023: 9
FWCI
24.83
Type
journal article
Language
en
Open access
Licence
Locations
  • Computers and Education Artificial Intelligence (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • NOAA Institutional Repository (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Science Foundation
  • National Oceanic and Atmospheric Administration
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
ChatGPT, large language models, Fundamentals of Engineering exam, environmental engineering education, artificial intelligence in education, exam question generation, mathematical problem solving, examination integrity, accessibility and inclusion

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review