The impact of text topic and assumed human vs. AI authorship on competence and quality assessment
Open accessAbstract
Background: While Large Language Models (LLMs) are considered positively with respect to technological progress and abilities, people are rather opposed to machines making moral decisions. But the circumstances under which algorithm aversion or algorithm appreciation are more likely to occur with respect to LLMs have not yet been sufficiently investigated. Therefore, the aim of this study was to investigate how texts with moral or technological topics, allegedly written either by a human author or by ChatGPT, are perceived. Methods: In a randomized controlled experiment, n = 164 participants read six texts, three of which had a moral and three a technological topic (predictor text topic). The alleged author of each text was randomly either labeled “ChatGPT” or “human author” (predictor authorship). We captured three dependent variables: assessment of author competence, assessment of content quality, and participants' intention to submit the text in a hypothetical university course (sharing intention). We hypothesized interaction effects, that is, we expected ChatGPT to score lower than alleged human authors for moral topics and higher than alleged human authors for technological topics and vice versa. Results: We only found a small interaction effect for perceived author competence, p = 0.004, d = 0.40, but not for the other dependent variables. However, ChatGPT was consistently devalued compared to alleged human authors across all dependent variables: there were main effects of authorship for assessment of the author competence, p < 0.001, d = 0.95; for assessment of content quality, p < 0.001, d = 0.39; as well as for sharing intention, p < 0.001, d = 0.57. There was also a small main effect of text topic on the assessment of text quality, p = 0.002, d = 0.35. Conclusion: These results are more in line with previous findings on algorithm aversion than with algorithm appreciation. We discuss the implications of these findings for the acceptance of the use of LLMs for text composition.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The findings offer partial support for the algorithm aversion hypothesis: participants reacted negatively to texts identified as AI-generated, but Version D received the highest mean scores despite containing AI-generated content, suggesting human editing obscured its AI origins and neutralized aversion; algorithm aversion is not a stable disposition but a contextually contingent response.
“Our findings offer partial support for the algorithm aversion hypothesis.”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Proksch et al. (2024)
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredProksch et al. (2024) reported that texts labeled as human-written were consistently rated higher in quality than identical texts labeled as AI-generated.
“Proksch et al. (2024) reported that texts labeled as human-written were consistently rated higher in quality than identical texts labeled as AI-generated”
From How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction (2026) · cites this paper as Proksch et al. (2024)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 2× · see citing works
“Moreover, they often tend to reproduce biases, lack transparency, and might even reduce critical thinking by their users (Kasneci et al., 2023).”
BackgroundDwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., et al. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102642 · Cited by 4,584 · see citing works
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · see citing works
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061 · see citing works
Emergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264 · see citing works
Cascella, M., Montomoli, J., Bellini, V., & Bignami, E. (2023) · Journal of Medical Systems · doi:10.1007/s10916-023-01925-4 · Cited by 1,120 · see citing works
Pavlik, J. V. (2023) · Journalism & Mass Communication Educator · doi:10.1177/10776958221149577 · Cited by 972 · cited 1× · see citing works
“ChatGPT can be used in almost any field, such as in the medical context (Cascella et al., 2023), in journalism (Pavlik, 2023), or in event planning (Keiper, 2023).”
BackgroundA systematic review of algorithm aversion in augmented decision making
Burton, J. W., Stein, M., & Jensen, T. B. (2020) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2155 · Cited by 814 · see citing works
People are averse to machines making moral decisions.
Bigman, Y. E., & Gray, K. (2018) · Cognition · doi:10.1016/j.cognition.2018.08.003 · Cited by 738 · see citing works
The benefits and challenges of chatgpt: An overview
Deng, J., & Lin, Y. (2023) · Frontiers in Computing and Intelligent Systems · doi:10.54097/fcis.v2i2.4465 · Cited by 611 · see citing works
ChatGPT and the rise of generative AI: Threat to academic integrity?
Eke, D. O. (2023) · Journal of Responsible Technology · doi:10.1016/j.jrt.2023.100060 · Cited by 532 · see citing works
Köbis, N., & Mossink, L. D. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106553 · Cited by 406 · see citing works
The Perceived Utility of Human and Automated Aids in a Visual Detection Task
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (2002) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720024494856 · Cited by 386 · see citing works
Co-constructing knowledge with generative AI tools: Reflections from a CSCL perspective
Creß, U., & Kimmerle, J. (2023) · International Journal of Computer-Supported Collaborative Learning · doi:10.1007/s11412-023-09409-w · Cited by 152 · see citing works
Self-citationHenestrosa, A. L., Greving, H., & Kimmerle, J. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107445 · Cited by 129 · see citing works
Self-citationHou, Y. T.-Y., & Jung, M. F. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479864 · Cited by 108 · see citing works
Dietvorst, B. J., & Bartels, D. M. (2021) · Journal of Consumer Psychology · doi:10.1002/jcpy.1266 · Cited by 96 · see citing works
ChatGPT in practice: Increasing event planning efficiency through artificial intelligence
Keiper, M. C. (2023) · Journal of Hospitality Leisure Sport & Tourism Education · doi:10.1016/j.jhlste.2023.100454 · Cited by 84 · see citing works
BOHM, R. M., Jörling, M., Reiter, L., & Fuchs, C. (2023) · Communications Psychology · doi:10.1038/s44271-023-00032-x · Cited by 55 · see citing works
Teaming Up with an AI: Exploring Human–AI Collaboration in a Writing Scenario with ChatGPT
Luther, T., Kimmerle, J., & Creß, U. (2024) · AI · doi:10.3390/ai5030065 · Cited by 55 · see citing works
Self-citationIn-Context Impersonation Reveals Large Language Models' Strengths and Biases
Salewski, L., Alaniz, S., Rio-Torto, I., Schulz, E., & Akata, Z. (2023) · neural information processing systems · doi:10.52202/075280-3152 · Cited by 51 · see citing works
Henestrosa, A. L., & Kimmerle, J. (2024) · Behavioral Sciences · doi:10.3390/bs14050353 · Cited by 24 · see citing works
Self-citationTHE IMPACT OF ARTIFICIAL INTELLIGENCE ON JOURNALISM. ADVERSE EFFECTS VS. BENEFITS
Stănescu, G. C. (2023) · RePEc: Research Papers in Economics · doi:10.5281/zenodo.8151134 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Effects of Assumed AI vs. Human Authorship on the Perception of a GPT-Generated Text
Henestrosa, A. L., & Kimmerle, J. (2024) · Journalism and Media · doi:10.3390/journalmedia5030069 · Cited by 52 · see citing works
Self-citationTeaming Up with an AI: Exploring Human–AI Collaboration in a Writing Scenario with ChatGPT
Luther, T., Kimmerle, J., & Creß, U. (2024) · AI · doi:10.3390/ai5030065 · Cited by 55 · see citing works
Self-citationThe impact of AI on education and careers: What do students think?
Thomson, S. R., Pickard-Jones, B., Baines, S., & Otermans, P. C. J. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1457299 · Cited by 43 · see citing works
Henestrosa, A. L., & Kimmerle, J. (2024) · Behavioral Sciences · doi:10.3390/bs14050353 · Cited by 24 · see citing works
Self-citationUnderstanding Reader Perception Shifts upon Disclosure of AI Authorship
Nakano, H., Takezawa, J., Matulic, F., Yang, C.-L., & Yatani, K. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789076 · Cited by 11 · see citing works
Li, Y., Gu, T., Yang, C., Li, M., Wang, C., Yao, L., Gu, W., & Sun, D. (2025) · Journal of Medical Internet Research · doi:10.2196/66161 · Cited by 5 · see citing works
Disclosing Generative AI Use in Digital Humanities Research
Ma, R., Zhang, X., & Wisnicki, A. S. (2025) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1471 · Cited by 4 · see citing works
AI aversion? Effects of author disclosure on young people’s perceptions of mental health advice
Brandtzæg, P. B., Skjuve, M. B., & Følstad, A. (2026) · Cyberpsychology Journal of Psychosocial Research on Cyberspace · doi:10.5817/cp2026-2-1 · Cited by 4 · see citing works
How Creative Is AI Writing? Generative and Collaborative AI in Japanese Fiction
Asano, Y., & Bastos, M. (2026) · Journal of Creative Communications · doi:10.1177/09732586261463934
Luther, T., Sommer, M., & Kimmerle, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93536-7_7
Self-citationDecision-making for technology-led transformation in research practices
Ulusoy, Ö. (2025) · AI & Society · doi:10.1007/s00146-025-02567-5
Javed, T., Azhar, J., Shabbir, M. F., & Nasir, A. (2025) · Discover Sustainability · doi:10.1007/s43621-025-01737-1
Dio, C. D., Manzi, F., Peretti, G., Longoni, F., D, V. G. M., Massaro, D., Gilli, G., & Marchetti, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5893627
Kimmerle, J., Ulsperger, A., & Creß, U. (2026) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2026.103339
Self-citationComparing Human and Virtual Influencers: How Self-Disclosure Shapes Engagement on InstagramPreprint
Sew, C., Gunasekara, S., Ma, Y., Teng, S., Genay, A., Kelly, R. M., & Gonçalves, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6172316
Yang, Y., Li, J., Liang, W., Peng, Y., Zou, Y., Yan, Z., Lyu, X., & Wang, Y. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2647127
A moral Turing test: How belief and source shape detection of and agreement with LLM judgments
Garcia, B., Qian, C., & Palminteri, S. (2026) · PLoS ONE · doi:10.1371/journal.pone.0353391
Dio, C. D., Manzi, F., Peretti, G., Longoni, F., D, V. G. M., Massaro, D., Gilli, G., & Marchetti, A. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101364
- Authors
- Proksch, S., Schühle, J., Streeb, E., Weymann, F., Luther, T., & Kimmerle, J.
- Affiliations
- University of Tübingen (DE)
- Leibniz-Institut für Wissensmedien (DE)
- Corresponding author
- Kimmerle, Joachim
- Venue
- Frontiers in Artificial Intelligence
- Publisher
- Frontiers Media SA
- Published
- 2024-05-31
- Volume
- 7
- Pages
- 1412710-1412710
- ISSN
- 2624-8212 (ISSN-L)
- PMC
- PMC11176609
- OpenAlex
- W4399246308
- Cited by
- 17
- References
- 23
- Citations by year
- 2026: 7 · 2025: 6 · 2024: 4
- FWCI
- 7.05
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Artificial Intelligence (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- OSF Preprints (OSF Preprints) (submittedVersion, cc-by)
- Funders
- Leibniz-Gemeinschaft
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Keywords
- algorithm aversion, algorithm appreciation, large language models, ChatGPT, authorship attribution, sharing intention
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
