CoChat
Sign Up
Cited by 106Artificial Intelligence in EducationArtificial Intelligence in Healthcare and EducationArtificial Intelligence Applications

Performance of ChatGPT on the US fundamentals of engineering exam: Comprehensive assessment of proficiency and potential implications for professional environmental engineering practice

Open access
Pursnani, Vinay, Sermet, Yusuf, Kurt, Musa, Demir, Ibrahim·2023·Computers and Education: Artificial Intelligence
DOI 10.1016/j.caeai.2023.100183
Open this paper in CoChat — free account

Abstract

In recent years, advancements in artificial intelligence (AI) have led to the development of large language models like GPT-4, demonstrating potential applications in various fields, including education. This study investigates the feasibility and effectiveness of using ChatGPT, a GPT-4 based model, in achieving satisfactory performance on the Fundamentals of Engineering (FE) Environmental Exam. This study further shows a significant improvement in the model's accuracy when answering FE exam questions through noninvasive prompt modifications, substantiating the utility of prompt modification as a viable approach to enhance AI performance in educational contexts. Furthermore, the findings reflect remarkable improvements in mathematical capabilities across successive iterations of ChatGPT models, showcasing their potential in solving complex engineering problems. Our paper also explores future research directions, emphasizing the importance of addressing AI challenges in education, enhancing accessibility and inclusion for diverse student populations, and developing AI-resistant exam questions to maintain examination integrity. By evaluating the performance of ChatGPT in the context of the FE Environmental Exam, this study contributes valuable insights into the potential applications and limitations of large language models in educational settings. As AI continues to evolve, these findings offer a foundation for further research into the responsible and effective integration of AI models across various disciplines, ultimately optimizing the learning experience and improving student outcomes.

Integrity signals

Registry

Crossref · 10.1016/j.caeai.2023.100183

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Pursnani et al. showed that ChatGPT's performance varied across different sections of the US Fundamentals of Engineering exam, performing exceptionally in Ethics and Professional practice while under-performing in Hydrology.

    “Similarly, Pursnani et al. evaluated ChatGPT’s performance in the context of Engineering, testing the model on the US Fundamentals of Engineering exam33. In their study, they showed that the model’s performance varied across different sections of the exam, performing exceptionally in areas such as Ethics and Professional practice, while under-performing in others such as Hydrology.”

    From Perception, performance, and detectability of conversational artificial intelligence across 32 university courses (2023) · cites this paper as [33]

Cites

Flagged first, then by citations

1 of the 33 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Pursnani, V., Sermet, Y., Kurt, M., & Demir, I.
Affiliations
University of Iowa (US)
Corresponding author
Sermet, Yusuf
ORCID
Venue
Computers and Education: Artificial Intelligence
Publisher
Elsevier BV
Published
2023
Volume
5
Pages
100183
ISSN
2666-920X (ISSN-L)
DOI
10.1016/j.caeai.2023.100183
OpenAlex
W4388490623
Cited by
106
References
33
Citations by year
2026: 13 · 2025: 34 · 2024: 50 · 2023: 9
FWCI
24.83
Type
journal article
Language
en
Open access
Licence
Locations
  • Computers and Education Artificial Intelligence (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • NOAA Institutional Repository (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Science Foundation
  • National Oceanic and Atmospheric Administration
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
ChatGPT, large language models, Fundamentals of Engineering exam, environmental engineering education, artificial intelligence in education, exam question generation, mathematical problem solving, examination integrity, accessibility and inclusion

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review