Using ChatGPT for writing articles for patients' education for dermatological diseases: A pilot study
Open accessThe DOI matches its Crossref record; 16 of the 17 references it cites were found in a registry or an index, and 1 are not checked yet; 2 of the 14 sources its claims cite were checked for support.
Abstract
Background: Patients’ education is a vital strategy for understanding a disease by patients and proper management of the condition. Physicians and academicians frequently make customized education materials for their patients. An artificial intelligence (AI)-based writer can help them write an article. Chat Generative Pre-Trained Transformer (ChatGPT) is a conversational language model developed by OpenAI (openai.com). The model can generate human-like responses. Objective: We aimed to evaluate the generated text from ChatGPT for its suitability in patients’ education. Materials and Methods: We asked the ChatGPT to list common dermatological diseases. It provided a list of 14 diseases. We used the disease names to converse with the application with disease-specific input (e.g., write a patient education guide on acne). The text was copied for checking the number of words, readability, and text similarity by software. The text’s accuracy was checked by a dermatologist following the structure of observed learning outcomes (SOLO) taxonomy. For the readability ease score, we compared the observed value with a score of 30. For the similarity index, we compared the observed value with 15% and tested it with a one-sample t-test. Results: The ChatGPT generated a paragraph of text of 377.43 ± 60.85 words for a patient education guide on skin diseases. The average text reading ease score was 46.94 ± 8.23 (P < 0.0001), and it indicates that this level of text can easily be understood by a high-school student to a newly joined college student. The text similarity index was higher (27.07 ± 11.46%, P = 0.002) than the expected limit of 15%. The text had a “relational” level of accuracy according to the SOLO taxonomy. Conclusion: In its current form, ChatGPT can generate a paragraph of text for patients’ educational purposes that can be easily understood. However, the similarity index is high. Hence, doctors should be cautious when using the text generated by ChatGPT and must check for text similarity before using it.
Verified 2026-10-06 · 1 reference not checked yet
Overview
CoChat Summary
TL;DR
This pilot cross-sectional study evaluated ChatGPT-generated patient education guides for common dermatological diseases and found the text readable but with high text similarity. The authors conclude that doctors should use ChatGPT-generated text cautiously and check for plagiarism before use.
Key findings
- ChatGPT listed 14 dermatological diseases/conditions: acne, atopic dermatitis, psoriasis, rosacea, shingles, vitiligo, warts, melanoma, basal cell carcinoma, squamous cell carcinoma, contact dermatitis, seborrheic dermatitis, ringworm, and cellulitis.
- The generated patient education guide averaged 377.43 ± 60.85 words, 23.07 ± 3.6 sentences, and a Flesch–Kincaid grade level of 10.53 ± 0.73; the Flesch Reading Ease score was 46.94 ± 8.23, significantly higher than the reference score of 30 (P < 0.0001).
- Overall text similarity was 27.07 ± 11.46%, significantly higher than the 15% cutoff (P = 0.002) and also higher than a 20% reference value (P = 0.038).
- Accuracy was rated as “relational” according to the SOLO taxonomy; word count did not correlate with ease score (r = 0.18, P = 0.54) or similarity index (r = −0.18, P = 0.55).
Methods
- Cross-sectional observational study conducted on January 23, 2023, over the Internet without human research participants; institutional ethics approval was not required according to prevailing country guidelines.
- ChatGPT version 9 was prompted with “Common skin diseases,” then with disease-specific inputs such as “write a patient education guide on acne”; generated text was copied for analysis.
- Word/sentence counts, Flesch–Kincaid Grade Level, and Flesch Reading Ease were calculated with an online calculator; text similarity was checked with Turnitin; accuracy was assessed by the third author using the SOLO taxonomy; one-sample t-tests compared ease score with 30 and similarity with 15%, with Pearson correlations also performed in GraphPad Prism 7.
Limitations
- The paper states this is a small-scale pilot study intended to initially identify issues authors may face when using AI-generated text; it calls for further studies with newer versions over time. It also states that ChatGPT text may be sourced from existing content and should be reviewed and edited by a human before publishing.
- Evident from the text: the study evaluated only 14 dermatological conditions, at a single time point and with one ChatGPT version; it did not test patient comprehension or clinical outcomes, used a single dermatologist for accuracy assessment, and relied on Turnitin for similarity, which may not capture all sources.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
2 Valid · 0 Not valid · 9 Not checked · 12 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This level of text can easily be understood by a high-school student to a newly joined college (grade level 10.53 ± 0.73) student without graduation.”
- Also stated in: § Results: · hedged
“it indicates that this level of text can easily be understood by a high-school student to a newly joined college student.”
- Also stated in: § Discussion · plain
“A student with the tenth standard onward can easily read and understand the topic written by ChatGPT.[12]”
§ Using ChatGPT for Writing Articles for Patients’ Education for Dermatological Diseases: A Pilot Study > Results
What the sources say
- Partly supportedcites [12] Kumar, 2013, The readability of editorials in popular Indian medical journals, Indian J Endocrinol Metab sourceExperimental — accuracy not yet measured
Kumar et al.'s INTRODUCTION (p. 1) supplies the interpretive rule the citing sentence leans on: 'The Flesch grade level rates the text on a US school grade level. For example, a grade of 10 indicates that a tenth grader (aged 15-17 year) can understand the document.' That supports the 'high-school student' reading of a grade level near 10, but the source says nothing about a 'newly joined college student without graduation' — an extension beyond its tenth-grade anchor. Nor does our copy report the value 10.53 ± 0.73: this paper's own measurements are Flesch readability scores of 26.1–34.8 for 64 editorials (Abstract Results; DISCUSSION, where text scoring up to 30 is described as 'best understood by graduates and above'), so that figure is the citing study's own, not this source's. (LLM, conf=0.72)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Results: · hedged
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This score was significantly higher (one-sample t-test P < 0.0001) than the average reading ease score of an editorial in Indian journals (expected ease score of 30).”
§ Using ChatGPT for Writing Articles for Patients’ Education for Dermatological Diseases: A Pilot Study > Results
What the sources say
- Partly supportedcites [12] Kumar, 2013, The readability of editorials in popular Indian medical journals, Indian J Endocrinol Metab sourceExperimental — accuracy not yet measured
Our copy does supply the benchmark this sentence leans on: the DISCUSSION states 'the mean Flesch score of all editorials was around 30 in the selected medical journals' and the CONCLUSION repeats that 'editorials in the popular medical and endocrinology journals from India have a readability score of 30'. However, the source reports no one-sample t-test against a reference value of 30 and no such significance statement; its MATERIALS AND METHODS describe only one-way ANOVA for between-group comparison and Spearman correlation, and the Abstract Results/DISCUSSION give P = 0.2666 for differences between journals, with P < 0.0001 attaching to correlations (Flesch score vs grade level, r2 = -0.9379; and score vs sentence count/words per sentence), not to a comparison with 30. Thus the 'expected ease score of 30' part is supported while the 'significantly higher (one-sample t-test P < 0.0001)' part is the citing authors' own analysis and is not stated in this source. (LLM, conf=0.72)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There was no correlation of word count with ease score (r = 0.18, P = 0.54) and similarity index (r = -0.18, P = 0.55).”
§ Using ChatGPT for Writing Articles for Patients’ Education for Dermatological Diseases: A Pilot Study > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“which is significantly (P = 0.002) higher than the commonly used cutoff by many of the journals (15%).”
- Also stated in: § Results: · plain
“The text similarity index was higher (27.07 ± 11.46%, P = 0.002) than the expected limit of 15%.”
§ Using ChatGPT for Writing Articles for Patients’ Education for Dermatological Diseases: A Pilot Study > Results
What the sources say
- Not checked yetcites [14] Q: What is the acceptable percentage of plagiarism report?. https://www.editage.com/insights/what-is-the-acceptable-percentage-of-plagiarism-report [Last accessed on 24 January 2023].
- Not checked yetcites [15] Plagiarism Scores. What the Plagiarism Score Means in the Context of the Student's Academic Submission. https://www.scanmyessay.com/plagiarism/interpret-plagiarism-score.php#:~:text=Anything%20above%2015%25%20is%20indicative, a%20range%20of%20disciplinary%20measures [Last accessed on 24 January 2023].
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Results: · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There was an average of 27.07 ± 11.46% of text similarity, which is significantly (P = 0.002) higher than the commonly used cutoff by many of the journals (15%).”
- Also stated in: § Results: · plain
“The text similarity index was higher (27.07 ± 11.46%, P = 0.002) than the expected limit of 15%.”
§ Using ChatGPT for Writing Articles for Patients’ Education for Dermatological Diseases: A Pilot Study > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Results: · plain
10 more claims
Create a free account to see all 23 claims and to verify the 12 references still open.
- Probably not a substantial claim. 8
- It may state two separate claims as one. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“ChatGPT-generated text may also have plagiarized content with variations in the percentage, sometimes beyond the tolerance level as it generates text from its training data.”
From ChatGPT in academic writing: Maximizing its benefits and minimizing the risks (2023) · cites this paper as [28]
Citations
Cites
ChatGPT listed as author on research papers: many scientists disapprove
Stokel-Walker, C. (2023) · Nature · doi:10.1038/d41586-023-00107-z · Cited by 868 · cited 1×
“However, controversies are continuing about whether the name ChatGPT should be included as an author or gets only acknowledgment.[6]”
BackgroundAbstracts written by ChatGPT fool scientists
Else, H. (2023) · Nature · doi:10.1038/d41586-023-00056-7 · Cited by 611 · cited 1×
“Hence, this software can generate human‐like text within seconds.[4]”
MethodNazari, N., Shabbir, M. S., & Setiawan, R. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e07014 · Cited by 405 · cited 1×
“483 Indian Dermatology Online Journal | Volume 14 | Issue 4 | July-August 2023 articles as it would write a paragraph of text in a minute, which is nearly impossible for a common researcher.[5] Some researchers are also including the ChatGPT as their coauthors as it is helping them to write the article as a coauthor h…”
BackgroundImproving Health Outcomes Through Patient Education and Partnerships with Patients
Paterick, T. E., Patel, N., Tajik, A. J., & Chandrasekaran, K. (2017) · Baylor University Medical Center Proceedings · doi:10.1080/08998280.2017.11929552 · Cited by 354 · cited 1×
“Patient's education is the process of providing information and instruction to patients and their families to help them understand and manage their health conditions.[1] This can include information about symptoms, treatment options, medications, and lifestyle changes that can improve their health.”
BackgroundChatGPT: Evolution or revolution?
Gordijn, B., & Have, H. T. (2023) · Medicine, Health Care and Philosophy · doi:10.1007/s11019-023-10136-0 · Cited by 317
Assessing reading levels of health information: uses and limitations of flesch formula
Jindal, P., & MacDermid, J. (2017) · Education for Health · doi:10.4103/1357-6283.210517 · Cited by 275 · cited 1×
“This calculates the Flesch–Kincaid Grade Level (indicates the years of education required to understand the text) and the Flesch Reading Ease Score (the higher the score, the easier it is to read).[8] The calculator also provides information on the total number of words, total sentences, average words per sentence, an…”
BackgroundTimmers, T., Janssen, L., Kool, R. B., & Kremer, J. A. (2020) · Journal of Medical Internet Research · doi:10.2196/17342 · Cited by 121 · cited 1×
“The goal of patient's education is to empower patients to take an active role in their own health care and improve their overall health outcomes.[2] Many physicians write their own article for dissemination among patients according to local needs.”
BackgroundThe Impact of Artificial Intelligence (AI) Programs on Writing Scientific Research.
Hammad, M. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03140-1 · Cited by 49 · cited 1×
“[17] AI can also assist with language editing and grammar checking, helping authors improve the clarity and readability of their writing.[18] Grammarly (https://www.”
BackgroundMemon, A. R. (2020) · Journal of Korean Medical Science · doi:10.3346/jkms.2020.35.e217 · Cited by 39 · cited 1×
“484 Indian Dermatology Online Journal | Volume 14 | Issue 4 | July-August 2023 journals have their right to select the percentage they tolerate,[13] a 15% text similarity is accepted by many of the journals.”
BackgroundBuchlak, Q. D., Esmaili, N., Bennett, C. C., & Farrokhi, F. R. (2021) · Acta neurochirurgica. Supplementum · doi:10.1007/978-3-030-85292-4_32 · Cited by 39 · cited 1×
“This could be used to generate reports, summaries, and other written materials from experimental data.[17] AI can also assist with language editing and grammar checking, helping authors improve the clarity and readability of their writing.”
BackgroundLucander, H., Bondemark, L., Brown, G., & Knutsson, K. (2010) · European Journal Of Dental Education · doi:10.1111/j.1600-0579.2009.00607.x · Cited by 38 · cited 1×
“The accuracy of the text was checked by the third author following the structure of observed learning outcomes (SOLO) taxonomy.[10] The study flow is shown in Figure 1.”
MethodConduct Common Statistical Tests Online
Mondal, H., Mondal, S., Majumder, R., & De, R. (2022) · Indian Dermatology Online Journal · doi:10.4103/idoj.idoj_605_21 · Cited by 27 · cited 1×
“The data were checked for normal distribution, and statistical tests were chosen according to the distribution (normal—parametric test, nonnormal— nonparametric tests).[11] As the data were normally distributed, we used a one‐sample t‐test for comparing the observed value with a (hypothetical) reference value.”
MethodSelf-citationArabyat, R. M., Qawasmeh, B. R., Al-Azzam, S. I., Nusair, M. B., & Alzoubi, K. H. (2022) · Journal of Empirical Research on Human Research Ethics · doi:10.1177/15562646221078655 · Cited by 23 · cited 1×
“This uses the same algorithm as iThenticate, which is used by the majority of journals worldwide to screen the articles for text similarity of submitted articles.[9] Data collection methods”
MethodBig Data/AI in Neurocritical Care: Maybe/Summary
Suárez, J. I. (2021) · Neurocritical Care · doi:10.1007/s12028-021-01422-x · Cited by 12 · cited 1×
“Additionally, AI‐powered tools can help researchers organize and analyze large amounts of data, making it easier to identify patterns and draw conclusions.[19] ChatGPT has the capability to do the task of grammar correction at a glance.”
BackgroundPeer review in scientific publishing: Current practice, guidelines, relevancy, and way forward
Mondal, S., Juhi, A., Kumari, A., Dhanvijay, A. K. D., Mittal, S., & Mondal, H. (2023) · Cosmoderma · doi:10.25259/csdm_35_2023 · Cited by 9 · cited 1×
“AI‐powered tools may help researchers to organize and analyze data, identify patterns and trends, and generate new hypotheses.[20] AI may also assist in the peer‐review process of academic papers, and this could be done by providing a pre‐review of the paper, flagging potential errors, or providing suggestions for imp…”
BackgroundSelf-citationThe researchers using AI to analyse peer review
Noorden, R. V. (2022) · Nature · doi:10.1038/d41586-022-02787-5 · Cited by 8
The readability of editorials in popular Indian medical journals
Kumar, K. V. S. H., Aravinda, K., & Varadarajulu, R. (2013) · Indian Journal of Endocrinology and Metabolism · doi:10.4103/2230-8210.119626 · Cited by 4
Research works printed but not linked: 1
- Artificial, 2020, Artificial Intelligence to help meet global demand for high-quality, objective peer-review in publishing., Artificial Intelligence to help
Cited by
Large language models in patient education: a scoping review of applications in medicine
Aydin, S., Karabacak, M., Vlachos, V., & Margetis, K. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1477898 · Cited by 288
Samala, A. D., Rawas, S., Wang, T., Reed, J. M., Kim, J., Howard, N.-J., & Ertz, M. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-12936-0 · Cited by 229
ChatGPT in medicine: prospects and challenges: a review article
Tan, S., Xin, X., & Wu, D. (2024) · International Journal of Surgery · doi:10.1097/js9.0000000000001312 · Cited by 126
“Research in periodontal disease, breast reconstruction, otolar-yngology, dermatology and other areas has also shown that ChatGPT is adept at addressing patients ’ concerns [7 – 10] .”
BackgroundKhan, N., Khan, Z., Koubaa, A., Khan, M. K., & Salleh, R. B. (2024) · Connection Science · doi:10.1080/09540091.2024.2353630 · Cited by 97
“However, the similarity index was higher than desired, indicating potential issues with text originality [58].”
BackgroundSaúde, S., Barros, J. P., & Almeida, I. (2024) · Social Sciences · doi:10.3390/socsci13080410 · Cited by 83
Metadata
- Authors
- Mondal, H., Mondal, S., & Podder, I.
- Affiliations
- College of Medicine & Sagore Dutta Hospital (IN)
- Corresponding author
- Mondal, Himel
- Mondal, Shaikat
- Venue
- Indian Dermatology Online Journal
- Published
- 2023
- Volume
- 14
- Issue
- 4
- Pages
- 482
- ISSN
- 2229-5178 (ISSN-L)
- PMC
- PMC10373821
- OpenAlex
- W4382774929
- Semantic Scholar
- f4efdae756c8704021e8acb2049f141ae6d0ad08
- Cited by
- 99
- References
- 17
- Citations by year
- 2026: 12 · 2025: 29 · 2024: 47 · 2023: 11
- FWCI
- 3.94
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by-nc-sa
- Locations
- Indian Dermatology Online Journal (publishedVersion, cc-by-nc-sa)
- PubMed Central (publishedVersion, cc-by-nc-sa)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Education › Artificial Intelligence in Education
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Keywords
- ChatGPT, large language models, patient education, skin diseases, readability, text similarity, SOLO taxonomy, AI-generated medical content, acne
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.