Issues in the 3rd year of the COVID-19 pandemic, including computer-based testing, study design, ChatGPT, journal metrics, and appreciation to reviewers
Open accessAbstract
After the appearance of ChatGPT-an artificial intelligence chatbot in November 2022, there may be submissions where the authors have received help from this chatbot, especially in the Introduction section.There is a debate on the authorship of ChatGPT [4].However, JEEHP will not consider to accept ChatGPT as an author, but ChatGPT-generated sentences or paragraphs may be cited.It is nearly impossible to detect whether ChatGPT generates text, because paragraphs generated using ChatGPT usually cannot be screened by plagiarism detection programs, including Similarity Check.Sources do not accompany ChatGPT-generated content.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Disapproval of including ChatGPT or any other LLM in the list of authors was explained in Science, Nature, and the Lancet editorials, which referred to such practice as scientific misconduct, and this view was echoed by many scientists.
“The disapproval of including ChatGPT or any other LLM in the list of authors was clearly explained in Science, Nature, and the Lancet editorials, which referred to such practice as scientific misconduct, and this view was echoed by many scientists [24,27,35,40,45].”
From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [35]
In “How to deal with manuscripts or paragraphs generated by ChatGPT” (page unavailable), Huh says JEEHP will not accept ChatGPT as an author. The References list a Nature article titled “ChatGPT listed as author on research papers: many scientists disapprove.” Our copy does not discuss Science or Lancet editorials, other LLMs, or classify such authorship as scientific misconduct. (LLM, conf=0.88)
Experimental — accuracy not yet measuredSeveral records advocated the need for proper and concise disclosure and documentation of ChatGPT or LLM use in the methodology or acknowledgement sections.
“In the case of ChatGPT use in the research process, several records advocated the need for the proper and concise disclosure and documentation of ChatGPT or LLM use in the methodology or acknowledgement sections [35,63,65].”
From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [35]
The section “How to deal with manuscripts or paragraphs generated by ChatGPT” (page not available) says ChatGPT-generated text may be cited and recommends checking its sources and accuracy. It does not advocate disclosing or documenting ChatGPT or LLM use in a methodology or acknowledgements section; searches for disclosure and acknowledgement guidance found no such passage in our copy. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 10 · Cited by 25
Research works printed but not linked: 9
- [1]Lee YS Presidential address: reflection on work from April 2019 to 2022 and appreciation to the staff and volunteers J Educ Eval Health Prof 2022 19 39 10.3352/jeehp.2022.19.39 36587955 PMC9922552 · DOI
- [2]Kim DH Kang YJ Park HK Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study J Educ Eval Health Prof 2022 19 33 10.3352/jeehp.2022.19.33 36503200 PMC9845067 · DOI
- [3]Park J Possibility of using the yes/no Angoff method as a substitute for the percent Angoff method for estimating the cutoff score of the Korean Medical Licensing Examination: a simulation study J Educ Eval Health Prof 2022 19 23 10.3352/jeehp.2022.19.23 36045595 PMC10086388 · DOI
- [5]Huh S Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study J Educ Eval Health Prof 2023 20 1 10.3352/jeehp.2023.20.1 36627845 PMC9905868 · DOI · Paper page
- [6]Huh S Emergence of the metaverse and ChatGPT in journal publishing after the COVID-19 pandemic Sci Ed 2023 10 1 4 10.6087/kcse.290 · DOI
- [7]Knipe D Jewkes R Equity in reporting settings of studies Eur Sci Ed 2022 48 e87545 10.3897/ese.2022.e87545 · DOI
- [8]Kim SY Suggestion of more suitable study designs and the corresponding reporting guidelines in articles published in the Journal of Educational Evaluation for Health Professions from 2021 to September 2022: a descriptive study J Educ Eval Health Prof 2022 19 36 10.3352/jeehp.2022.19.36 36567310 PMC9889887 · DOI
- [9]Huh S Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers J Educ Eval Health Prof 2022 19 2 10.3352/jeehp.2022.19.2 35021317 PMC8853862 · DOI
- [10]Eisen MB Akhmanova A Behrens TE Diedrichsen J Harper DM Iordanova MD Weigel D Zaidi M Peer review without gatekeeping Elife 2022 11 e83889 10.7554/eLife.83889 36263932 PMC9584602 · DOI
- Authors
- Huh, S.
- Affiliations
- Hallym University (KR)
- Corresponding author
- Huh, Sun
- ORCID
- Huh, Sun
- Venue
- Journal of Educational Evaluation for Health Professions
- Published
- 2023-01-31
- Volume
- 20
- Pages
- 5
- ISSN
- 1738-1339 (ISSN-L)
- 1975-5937
- PMC
- PMC9986465
- OpenAlex
- W4318592207
- Cited by
- 25
- References
- 10
- Citations by year
- 2025: 3 · 2024: 8 · 2023: 14
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Journal of Educational Evaluation for Health Professions (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Innovations in Medical Education
- Keywords
- ChatGPT, plagiarism detection, computer-based testing, research design, journal metrics, COVID-19 pandemic
- MeSH terms
- COVID-19, SARS-CoV-2, Humans, Bibliometrics, Peer Review, Research, Benchmarking, Pandemics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
