Co-Writing with Opinionated Language Models Affects Users’ Views
Open accessThe DOI matches its Crossref record; 130 of the 130 references it cites were found in a registry or an index; 12 of the 28 sources its claims cite were checked for support.
Abstract
If large language models like GPT-3 preferably produce a particular point of view, they may influence people’s opinions on an unknown scale. This study investigates whether a language-model-powered writing assistant that generates some opinions more often than others impacts what users write – and what they think. In an online experiment, we asked participants (N=1,506) to write a post discussing whether social media is good for society. Treatment group participants used a language-model-powered writing assistant configured to argue that social media is good or bad for society. Participants then completed a social media attitude survey, and independent judges (N=500) evaluated the opinions expressed in their writing. Using the opinionated language model affected the opinions expressed in participants’ writing and shifted their opinions in the subsequent attitude survey. We discuss the wider implications of our results and argue that the opinions built into AI language technologies need to be monitored and engineered more carefully.
Overview
CoChat Summary
TL;DR
In an online experiment, participants co-writing with a GPT-3 assistant prompted to argue that social media is good or bad were more likely to express that view in their posts and shifted in a later attitude survey. The authors call this “latent persuasion by language models” and argue that model opinions need to be monitored and engineered more carefully.
Key findings
- Participants using a model supportive of social media were 2.04 times more likely than controls to argue social media is good for society (p<0.0001, 95% CI [1.83, 2.30]); those using a critical model were 2.0 times more likely to argue it is bad (p<0.0001, 95% CI [1.79, 2.24]).
- In the post-task attitude survey, 45% of participants with the supportive model said social media is good for society, compared with 35% of controls (d=0.22, p<0.001); the critical model shifted views toward “bad” (d=0.19, p<0.005).
- Most participants did not blindly accept suggestions: on average 63% of sentences were written by participants themselves, 25% were mixed, and 11.5% were fully accepted from the model; about one in four accepted no suggestion, and one in ten had more than 75% of their post written by the model.
- Awareness was limited: only 10% of participants whose views aligned with the model noticed imbalanced suggestions, while 30% of those contradicted by the model noticed skew; about 20% of non-aligned participants believed the model influenced them.
Methods
- Online experiment with N=1,506 post-exclusion participants (507 control, 508 techno-optimist, 491 techno-pessimist), recruited via Prolific as US-based adults at least 18 years old; participants wrote at least five sentences replying to a Reddit-style post on whether social media is good for society.
- Treatment groups used a GPT-3 (“text-davinci-002”) writing assistant configured via prefix and infix prompts to generate suggestions arguing social media is good or bad; controls wrote without assistance.
- Outcomes included sentence-level opinion labels from N=500 judges (84.1% agreement, Cohen’s κ=0.76), keystroke-level interaction logs, a post-task attitude survey, and a user-experience survey; analysis used logistic regression and treated the ordinal opinion scale as interval (-1 bad, 1 good, 0 both/neither).
Limitations
- The paper states that the study tested only one topic, one specific GPT-3 writing-assistant implementation, and a strongly opinionated model; generalization to entrenched topics, other applications, and weaker models is unknown, and persistence over time was not tested.
- The paper states that pre-task opinion was measured retrospectively and is biased by the treatment effect, amounting to 14% of its standard deviation; the sample was US-based Prolific participants doing a short paid task (average 5.9 minutes).
- The paper states that demand effects cannot be fully ruled out, though most participants did not identify the study’s purpose as testing the assistant’s influence; the mechanisms behind the observed opinion shift remain unidentified.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
8 Valid · 1 Not valid · 8 Not checked · 81 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“We also experimented with fine-tuning [53] to guide the models' opinion, but the fine-tuned models did not consistently produce the intended opinion.”
What the sources say
- Supportedcites [53] [53] Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146 (2018). - source
The citation is attached to “fine-tuning,” identifying the method the citing authors tried. The source’s Introduction describes its proposed Universal Language Model Fine-tuning method. The statement that the citing authors’ models did not consistently produce the intended opinion reports their own experiment, not a finding attributed to this source. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“when participants strongly argued for another opinion in their writing, the model's continuations would follow their opinion”
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“shifted their opinions in the subsequent attitude survey”
- Also stated in: § 1 INTRODUCTION · p. 2 · plain
“We find that both participants' writing and their attitude towards social media in the survey were considerably affected by the model's preferred opinion.”
plainstates a causen = 1,506- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 INTRODUCTION · p. 2 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This additional prompt guided the model's continuation towards the target opinion even if participants had articulated a different opinion earlier in their writing.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using the opinionated language model affected the opinions expressed in participants' writing”
- Also stated in: § 1 INTRODUCTION · p. 2 · plain
“We find that both participants' writing and their attitude towards social media in the survey were considerably affected by the model's preferred opinion.”
plainstates a causen = 1,506- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 INTRODUCTION · p. 2 · plain
43 more claims
Create a free account to see all 98 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 36
- It may state two separate claims as one. 19
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“More recently, [29] found that writing with AI assistance on opinionated content affects not only a user's writing output but also their own opinions toward the written topic.”
From 'It was 80% me, 20% AI': Seeking Authenticity in Co-Writing with Large Language Models (2025) · cites this paper as [29]
The cited source says what the claim attributes to it.
In the citing paper
“Furthermore, using a large language model has been shown to affect the user's own opinions [102].”
From The Robots Are Here: Navigating the Generative AI Revolution in Computing Education (2023) · cites this paper as [102]
The cited source says what the claim attributes to it.
In the citing paper
“In fact, they found that suggestion systems influence what people write, even when suggestions do not reflect their own opinions (see also [58]).”
From The AI Ghostwriter Effect: When Users do not Perceive Ownership of AI-Generated Text but Self-Declare as Authors (2024) · cites this paper as [58]
The cited source says what the claim attributes to it.
In the citing paper
“On the other hand, LLMs can influence and persuade humans26, with research demonstrating that LLM-based writing assistants are capable of swaying people’s views27.”
From Generative language models exhibit social identity biases (2024) · cites this paper as [27]
The cited source says what the claim attributes to it.
In the citing paper
“Writers who compose with an opinionated language model shift their expressed views toward the model's (Jakesch et al., 2023), and writers who accept substantial AI contributions report reduced ownership of the resulting text (Draxler et al., 2024).”
From Think Before You Accept: Can Written Justification Reduce Uncritical Uptake of AI Writing Suggestions? (2026) · cites this paper as Jakesch et al., 2023
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Bem, D. J. (1972) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60024-6 · Cited by 5,241 · cited 1×
“According to self-perception theory [13], such changes in behavior may lead to changes in opinion.”
The Spread of Obesity in a Large Social Network over 32 Years
Christakis, N. A., & Fowler, J. H. (2007) · New England Journal of Medicine · doi:10.1056/nejmsa066082 · Cited by 5,007
Behavioral Study of obedience.
Milgram, S. (1963) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0040525 · Cited by 4,918 · cited 1×
“On a personal level, people may conform to majority views against their better judgement [6] and obey an authority figure even if it means harming others [80].”
BackgroundI tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience
Marwick, A. E., & boyd, D. (2011) · New Media & Society · doi:10.1177/1461444810365313 · Cited by 4,612 · cited 1×
“Research has shown that a variety of sources infuences users’ attitudes and behaviors, including friends, family, experts, and internet celebrities [48, 78]; the latter group was labeled infuencers due to their infuence on a large group of ’followers’ [10].”
BackgroundHumans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554
Universal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1031 · Cited by 4,100 · cited 1×
“We also experimented with fine-tuning [53] to guide the models’ opinion, but the fine-tuned models did not consistently produce the intended opinion.”
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · arXiv · doi:10.48550/arxiv.1906.02243 · Cited by 3,735
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1355 · Cited by 3,684 · cited 1×
“However, more cautious voices have also warned about the ethical and social risks of harm from large language models [100, 101], ranging from discrimination and exclusion [23, 54, 83] to misinformation [67, 75, 91, 106] and environmental [96] and socioeconomic harms [14].”
BackgroundProlific.ac—A subject pool for online experiments
Palan, S., & Schitter, C. (2018) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2017.12.004 · Cited by 3,450
The Elaboration Likelihood Model of Persuasion
Petty, R. E., & Cacioppo, J. T. (1986) · Communication and Persuasion · doi:10.1007/978-1-4612-4964-1_1 · Cited by 3,202
The People's Choice: How the Voter Makes Up His Mind in a Presidential Campaign
Lazarsfeld, P., Berelson, B., & Gaudet, H. (1968) · doi:10.7312/laza93930 · Cited by 2,905
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · cited 1×
“Prior work on the societal risks of large language models has warned that models learn stereotypes and biases from their training data [14, 28, 44] that may be amplified through widespread deployments [19].”
BackgroundOn the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · cited 9×
“guage models [21, 97, 103], there has been a growing interest in design goals beyond efciency.”
BackgroundInfluentialEffects of Group Pressure Upon the Modification and Distortion of Judgments
Asch, S. E. (1961) · Documents of Gestalt Psychology · doi:10.1525/9780520313514-017 · Cited by 2,644
The collective dynamics of smoking in a large social network
Christakis, N. A., & Fowler, J. H. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa0706154 · Cited by 2,390 · cited 1×
“On a societal level, researchers have shown that social infuence drives speculative markets [93], afects voting patterns [69], and contributes to the spread of unhealthy behaviors such as smoking and obesity [30, 31].”
Background(2017) · Managerial Leadership for Librarians · doi:10.5040/9798400682018.ch-003 · Cited by 2,240
Nass, C., Steuer, J., & Tauber, E. R. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191703 · Cited by 2,148 · cited 2×
“As appropriate for an early study, our experiment has several limitations: We only tested whether a language model affected participants’ views on a single topic.”
The echo chamber effect on social media
Cinelli, M., De Francisci Morales, G., Galeazzi, A., Quattrociocchi, W., & Starnini, M. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2023301118 · Cited by 2,092 · cited 1×
“Even without being designed to shift opinions, it has been found that algorithms may contribute to political polarization by reinforcing divisive opinions [9, 24, 32].”
BackgroundAlgorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daum'e, H., & Wallach, H. (2020) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2005.14050 · Cited by 1,975
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016) · Communications of the ACM · doi:10.1145/2818717 · Cited by 1,973
Exposure to opposing views on social media can increase political polarization
Bail, C. A., Argyle, L. P., Brown, T. W., Bumpus, J. P., Chen, H., Hunzaker, M. B. F., Lee, J., Mann, M., Merhout, F., & Volfovsky, A. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1804840115 · Cited by 1,863 · cited 1×
“Even without being designed to shift opinions, it has been found that algorithms may contribute to political polarization by reinforcing divisive opinions [9, 24, 32].”
BackgroundComplacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710
Everyone's an influencer: quantifying influence on twitter
Bakshy, E., Hofman, J. M., Mason, W. A., & Watts, D. J. (2011) · Proceedings of the fourth ACM international conference on Web search and data mining · doi:10.1145/1935826.1935845 · Cited by 1,689 · cited 1×
“Research has shown that a variety of sources influences users’ attitudes and behaviors, including friends, family, experts, and internet celebrities [48, 78]; the latter group was labeled influencers due to their influence on a large group of ’followers’ [10].”
BackgroundPersuasive technology: using computers to change what we think and do
Fogg, B. J. (2002) · Ubiquity · doi:10.1145/764008.763957 · Cited by 1,554
Nudge: Improving Decisions About Health, Wealth, and Happiness
Hoyt, G. M. (2009) · International Review of Economics Education · doi:10.1016/s1477-3880(15)30073-6 · Cited by 1,142
Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050 · cited 1×
“Prior work on the societal risks of large language models has warned that models learn stereotypes and biases from their training data [14, 28, 44] that may be amplified through widespread deployments [19].”
BackgroundIn AI we trust? Perceptions about automated decision-making by artificial intelligence
Araujo, T., Helberger, N., Kruikemeier, S., & de Vreese, C. H. (2020) · AI & SOCIETY · doi:10.1007/s00146-019-00931-w · Cited by 1,038
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.229 · Cited by 904
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2021) · arXiv · doi:10.48550/arxiv.2109.07958 · Cited by 899 · cited 1×
“However, more cautious voices have also warned about the ethical and social risks of harm from large language models [100, 101], ranging from discrimination and exclusion [23, 54, 83] to misinformation [67, 75, 91, 106] and environmental [96] and socioeconomic harms [14].”
BackgroundTaxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · cited 1×
“However, more cautious voices have also warned about the ethical and social risks of harm from large language models [100, 101], ranging from discrimination and exclusion [23, 54, 83] to misinformation [67, 75, 91, 106] and environmental [96] and socioeconomic harms [14].”
BackgroundAI-Mediated Communication: Definition, Research Agenda, and Ethical Considerations
Hancock, J. T., Naaman, M., & Levy, K. (2020) · Journal of Computer-Mediated Communication · doi:10.1093/jcmc/zmz022 · Cited by 658 · cited 1×
“Applications like writing assistants [38], grammar support [66], and machine translation [45] inject the models’ output into what people write and read [51].”
BackgroundSelf-citationRole of the New Media in the Arab Spring
Khondker, H. H. (2011) · Globalizations · doi:10.1080/14747731.2011.621287 · Cited by 630 · cited 1×
“After initial excitement about social media’s democratic potential [62], it became evident that technologies that affect public opinion will be subject to powerful political and commercial interests [22].”
BackgroundMedia Technologies: Essays on Communication, Materiality, and Society
(2014) · The MIT Press eBooks · doi:10.7551/mitpress/9780262525374.001.0001 · Cited by 580
Nass, C., Steuer, J., & Tauber, E. R. (1994) · Conference companion on Human factors in computing systems - CHI '94 · doi:10.1145/259963.260288 · Cited by 579
Defending Against Neural Fake News
Zellers, R., Holtzman, A., Rashkin, H., Bisk, Y., Farhadi, A., Roesner, F., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1905.12616 · Cited by 569
Political Effects of the Internet and Social Media
Zhuravskaya, E., Petrova, M., & Enikolopov, R. (2020) · Annual Review of Economics · doi:10.1146/annurev-economics-081919-050239 · Cited by 560
Troops, Trolls and Troublemakers: A Global Inventory of Organized Social Media Manipulation
Bradshaw, S., & Howard, P. (2017) · Oxford University Research Archive (ORA) (University of Oxford) · Cited by 543
Ethical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477 · cited 1×
“However, more cautious voices have also warned about the ethical and social risks of harm from large language models [100, 101], ranging from discrimination and exclusion [23, 54, 83] to misinformation [67, 75, 91, 106] and environmental [96] and socioeconomic harms [14].”
BackgroundIs seeing believing?: how recommender system interfaces affect users' opinions
Cosley, D., Lam, S. K., Albert, I., Konstan, J. A., & Riedl, J. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642713 · Cited by 467 · cited 1×
“Studies have evaluated the influence that technical artifacts such as personalized recommendations, chatbots, and choice architectures have on users’ decision-making [15, 35, 50, 72].”
BackgroundAdvances in Experimental Social Psychology, Vol. 19
Fine, G. A., & Berkowitz, L. (1987) · Contemporary Sociology · doi:10.2307/2070758 · Cited by 458
Koltovskaia, S. (2020) · Assessing Writing · doi:10.1016/j.asw.2020.100450 · Cited by 447
The structure of online diffusion networks
Goel, S., Watts, D. J., & Goldstein, D. G. (2012) · Proceedings of the 13th ACM Conference on Electronic Commerce · doi:10.1145/2229012.2229058 · Cited by 385 · cited 1×
“Research has shown that a variety of sources influences users’ attitudes and behaviors, including friends, family, experts, and internet celebrities [48, 78]; the latter group was labeled influencers due to their influence on a large group of ’followers’ [10].”
BackgroundTreating ordinal scales as interval scales: an attempt to resolve the controversy.
Knapp, T. R. (1990) · PubMed · Cited by 375
Wordcraft: Story Writing With Large Language Models
Yuan, A., Coenen, A., Reif, E., & Ippolito, D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511105 · Cited by 374
(2019) · Cited by 369
Advertising in social media: a review of empirical evidence
Knoll, J. (2016) · International Journal of Advertising · doi:10.1080/02650487.2015.1021898 · Cited by 362 · cited 1×
“Like search engine and social media network operators [65], operators of these applications may choose to monetize the persuasive power of their technology.”
BackgroundAll the News That’s Fit to Fabricate: AI-Generated Text as a Tool of Media Misinformation
Kreps, S., McCain, R. M., & Brundage, M. (2022) · Journal of Experimental Political Science · doi:10.1017/xps.2020.37 · Cited by 360 · cited 2×
“However, more cautious voices have also warned about the ethical and social risks of harm from large language models [100, 101], ranging from discrimination and exclusion [23, 54, 83] to misinformation [67, 75, 91, 106] and environmental [96] and socioeconomic harms [14].”
BackgroundLee, M., Liang, P., & Yang, Q. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502030 · Cited by 345
Human heuristics for AI-generated language are flawed
Jakesch, M., Hancock, J. T., & Naaman, M. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2208839120 · Cited by 342
Self-citationThe Anatomy of Reddit: An Overview of Academic Research
Medvedev, A. N., Lambiotte, R., & Delvenne, J. (2019) · Springer proceedings in complexity · doi:10.1007/978-3-030-14683-2_9 · Cited by 276 · cited 1×
“Such discussions have a large readership [79], pertain to political controversies, and are plausible application settings for writing assistants and language models.”
BackgroundTreating Ordinal Scales as Interval Scales: An Attempt To Resolve the Controversy
KNAPP, T. R. (1990) · Nursing Research · doi:10.1097/00006199-199003000-00019 · Cited by 271 · cited 1×
“Since the opinion scale has comparable-size intervals and a zero point, continuous analysis is meaningful and justifable [64].”
BackgroundCreative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257 · cited 1×
“Studies have investigated interface design factors and interactions with writing assistants that directly or indirectly support inspiration [17, 70, 95, 105], language profciency [27], story writing [95, 105], text revision [36, 107] or creative writing [33, 46].”
BackgroundScaling Language Models: Methods, Analysis & Insights from Training Gopher
Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.11446 · Cited by 235 · cited 3×
“However, more cautious voices have also warned about the ethical and social risks of harm from large language models [100, 101], ranging from discrimination and exclusion [23, 54, 83] to misinformation [67, 75, 91, 106] and environmental [96] and socioeconomic harms [14].”
BackgroundAI as a moral crumple zone: The effects of AI-mediated communication on attribution and trust
Hohenstein, J., & Jung, M. (2020) · Computers in Human Behavior · doi:10.1016/j.chb.2019.106190 · Cited by 212 · cited 1×
“Furthermore, the sentiment of the suggested text may afect the sentiment of the written text [3, 52].”
BackgroundProtecting elections from social media manipulation
Aral, S., & Eckles, D. (2019) · Science · doi:10.1126/science.aaw8243 · Cited by 209
EFFECTS OF GROUP PRESSURE UPON THE MODIFICATION AND DISTORTION OF JUDGMENTS
Asch, S. E. (2023) · Documents of Gestalt Psychology · doi:10.2307/jj.5233080.20 · Cited by 206
Political Effects of the Internet and Social Media
Zhuravskaya, E., Petrova, M., & Enikolopov, R. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3439957 · Cited by 199 · cited 1×
“A related topic, the political repercussions of social media and recommender systems [108], has received extensive research attention, however.”
BackgroundGarrido-Muñoz, I., Montejo-Ráez, A., Santiago, F. M., & Ureña-López, L. (2021) · Applied Sciences · doi:10.20944/preprints202103.0049.v1 · Cited by 191
How do People Type on Mobile Devices?: Observations from a Study with 37,000 Volunteers
Palin, K., Feit, A. M., Kim, S., Kristensson, P. O., & Oulasvirta, A. (2019) · Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/3338286.3340120 · Cited by 190
Smart Reply: Automated Response Suggestion for Email
Kannan, A., Kurach, K., Ravi, S., Kaufmann, T., Tomkins, A., Miklos, B., Corrado, G., Lukacs, L., Ganea, M., Young, P., et al. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939801 · Cited by 183
Language Models are Few-shot Multilingual Learners
Winata, G. I., Madotto, A., Lin, Z., Liu, R., Yosinski, J., & Fung, P. (2021) · MRL · doi:10.48550/arxiv.2109.07684 · Cited by 182
Complacency and Automation Bias in the Use of Imperfect Automation
Wickens, C. D., Clegg, B. A., Vieane, A. Z., & Sebok, A. L. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815581940 · Cited by 176
Rashotte, L. (2007) · The Blackwell Encyclopedia of Sociology · doi:10.1002/9781405165518.wbeoss154 · Cited by 171
Gmail Smart Compose: Real-Time Assisted Writing
Chen, M. X., Lee, B. N., Bansal, G., Cao, Y., Zhang, S., Lu, J., Tsay, J., Wang, Y., Dai, A. M., Chen, Z., et al. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330723 · Cited by 169
Becker, C. B., Smith, L. M., & Ciao, A. C. (2006) · Journal of Counseling Psychology · doi:10.1037/0022-0167.53.4.550 · Cited by 167 · cited 1×
“Even participants with preformed opinions on the topic may have changed their attitudes by being encouraged to communicate a belief that runs counter to their own belief [12, 99].”
BackgroundKarinshak, E., Liu, S. X., Park, J. S., & Hancock, J. T. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579592 · Cited by 167
Group forces in the modification and distortion of judgments.
Asch, S. E. (2004) · Social psychology. · doi:10.1037/10025-016 · Cited by 161
WatchWriter: Tap and Gesture Typing on a Smartwatch Miniature Keyboard with Statistical Decoding
Gordon, M., Ouyang, T., & Zhai, S. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858242 · Cited by 150 · cited 1×
“They suggest one to three subsequent words based on underlying likelihood distributions [40, 43, 49, 89].”
BackgroundMetaphoria: An Algorithmic Companion for Metaphor Creation
Gero, K. I., & Chilton, L. B. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300526 · Cited by 146 · cited 1×
“Studies have investigated interface design factors and interactions with writing assistants that directly or indirectly support inspiration [17, 70, 95, 105], language profciency [27], story writing [95, 105], text revision [36, 107] or creative writing [33, 46].”
BackgroundA General Language Assistant as a Laboratory for Alignment
Askell, A., Bai, Y., Chen, A., Drain, D., Ganguli, D., Henighan, T., Jones, A., Joseph, N., Mann, B., DasSarma, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.00861 · Cited by 146
Humans rely more on algorithms than social influence as a task becomes more difficult
Bogert, E., Schecter, A., & Watson, R. T. (2021) · Scientific Reports · doi:10.1038/s41598-021-87480-9 · Cited by 144
Vertanen, K., Memmi, H., Emge, J., Reyal, S., & Kristensson, P. O. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702135 · Cited by 133 · cited 1×
“Typical text entry systems attend to language context at the word [18, 98] or sentence level [5, 27].”
BackgroundEffects of Language Modeling and its Personalization on Touchscreen Typing Performance
Fowler, A., Partridge, K., Chelba, C., Bi, X., Ouyang, T., & Zhai, S. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702503 · Cited by 127 · cited 1×
“They suggest one to three subsequent words based on underlying likelihood distributions [40, 43, 49, 89].”
BackgroundReducing Sentiment Bias in Language Models via Counterfactual Evaluation
Huang, P.-S., Zhang, H., Jiang, R., Stanforth, R., Welbl, J., Rae, J., Maini, V., Yogatama, D., & Kohli, P. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.7 · Cited by 124 · cited 3×
“However, more cautious voices have also warned about the ethical and social risks of harm from large language models [100, 101], ranging from discrimination and exclusion [23, 54, 83] to misinformation [67, 75, 91, 106] and environmental [96] and socioeconomic harms [14].”
BackgroundGarrido-Muñoz, I., Montejo-Ráez, A., Martínez-Santiago, F., & Ureña-López, L. A. (2021) · Applied Sciences · doi:10.3390/app11073184 · Cited by 123
Influencing Individually: Fusing Personalization and Persuasion
(2012) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2209310.2209312 · Cited by 119
Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence
Singh, N., Bernal, G., Savchenko, D., & Glassman, E. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3511599 · Cited by 119 · cited 2×
“Studies have investigated interface design factors and interactions with writing assistants that directly or indirectly support inspiration [17, 70, 95, 105], language prof-ciency [27], story writing [95, 105], text revision [36, 107] or creative writing [33, 46].”
BackgroundACGS-2: A Production-Ready Constitutional AI Governance System
Lyu, H. (2025) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.16416793 · Cited by 119
How Different Groups Prioritize Ethical Values for Responsible AI
Jakesch, M., Buçinca, Z., Amershi, S., & Olteanu, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533097 · Cited by 116 · cited 1×
“[55] has highlighted that the values held by those developing AI systems difer from those of the broader”
BackgroundSelf-citationBuschek, D., Zürn, M., & Eiband, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445372 · Cited by 108
Self-citation(2015) · Choice Reviews Online · doi:10.5860/choice.190563 · Cited by 108
Dunlop, M., & Levine, J. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208659 · Cited by 99
A Cost-Benefit Study of Text Entry Suggestion Interaction
Quinn, P., & Zhai, S. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858305 · Cited by 98
HONEST: Measuring Hurtful Sentence Completion in Language Models
Nozza, D., Bianchi, F., & Hovy, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.191 · Cited by 95 · cited 3×
“However, more cautious voices have also warned about the ethical and social risks of harm from large language models [100, 101], ranging from discrimination and exclusion [23, 54, 83] to misinformation [67, 75, 91, 106] and environmental [96] and socioeconomic harms [14].”
BackgroundLanguage Models are Few-shot Multilingual Learners
Winata, G. I., Madotto, A., Lin, Z., Liu, R., Yosinski, J., & Fung, P. (2021) · Proceedings of the 1st Workshop on Multilingual Representation Learning · doi:10.18653/v1/2021.mrl-1.1 · Cited by 93 · cited 4×
“It is known that language models work and perform better for the languages and contexts they are trained in (primarily English or Mandarin Chinese) [23, 91, 103].”
BackgroundMethodInfluential”Because AI is 100% right and safe”: User Attitudes and Sources of AI Authority in India
Kapania, S., Siy, O., Clapper, G., SP, A. M., & Sambasivan, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517533 · Cited by 90 · cited 2×
“We also provided them with a link to a website with a nonpartisan overview of the pros and cons of social media and asked them whether they had comments about the research.”
BackgroundMethodPredictive text encourages predictable writing
Arnold, K. C., Chauncey, K., & Gajos, K. Z. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377523 · Cited by 87
Zhang, M. R., Wen, H., & Wobbrock, J. O. (2019) · Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3332165.3347924 · Cited by 86
Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries
Dang, H., Benharrak, K., Lehmann, F., & Buschek, D. (2022) · Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3526113.3545672 · Cited by 84
Self-citationIs seeing believing? how recommender system interfaces affect users' opinions
Cosley, D., Lam, S. K., Albert, I., Konstan, J. A., & Riedl, J. (2003) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/642712.642713 · Cited by 81
The Persuasive Power of Algorithmic and Crowdsourced Advice
Gunaratne, J., Zalmanson, L., & Nov, O. (2018) · Journal of Management Information Systems · doi:10.1080/07421222.2018.1523534 · Cited by 80 · cited 2×
“The influence that algorithmic entities have on people depends on how people perceive the algorithm, for example, whether they attribute trustworthiness to its recommendations [50, 76].”
BackgroundSelf-citationUniversal Language Model Fine-Tuning for Text Classification
Reddy, R., V (2023) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2023.53887 · Cited by 75
Ethics in cybersecurity research and practice
Macnish, K., & van der Ham, J. (2020) · Technology in Society · doi:10.1016/j.techsoc.2020.101382 · Cited by 73 · cited 1×
“While risky, decisions to share vulnerabilities have led to positive developments in computer safety [77].”
BackgroundOn Suggesting Phrases vs. Predicting Words for Mobile Text Composition
Arnold, K. C., Gajos, K. Z., & Kalai, A. T. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984511.2984584 · Cited by 69
All the News that’s Fit to Fabricate: AI-Generated Text as a Tool of Media MisinformationPreprint
Kreps, S. E., McCain, R. M., & Brundage, M. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3525002 · Cited by 66
Both complete and correct?: multi-objective optimization of touchscreen keyboard
Bi, X., Ouyang, T., & Zhai, S. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557414 · Cited by 59
ResearchIME: A Mobile Keyboard Application for Studying Free Typing Behaviour in the Wild
Buschek, D., Bisinger, B., & Alt, F. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173829 · Cited by 51 · cited 1×
“tures such as auto-correction and word suggestions can negatively impact typing performance and user experience [11, 26, 37, 85].”
BackgroundSelf-citationThe Ghost in the Machine has an American accent: value conflict in GPT-3
Johnson, R. L., Pistilli, G., Menédez-González, N., Duran, L. D. D., Panai, E., Kalpokiene, J., & Bertulfo, D. J. (2022) · arXiv · doi:10.48550/arxiv.2203.07785 · Cited by 50 · cited 3×
“Small-n audits have also suggested that the values embedded in models like GPT-3 were more aligned with reported dominant US values than those upheld in other cultures [57].”
BackgroundBhat, A., Agashe, S., Oberoi, P., Mohile, N., Jangir, R., & Joshi, A. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584060 · Cited by 47 · cited 5×
“[17] discuss how writers evaluate the suggestions provided and integrate them into diferent cognitive writing processes.”
BackgroundInfluentialSelf-citationBhat, A., Agashe, S., Mohile, N., Oberoi, P., Jangir, R., & Joshi, A. (2022) · arXiv · doi:10.48550/arxiv.2208.00636 · Cited by 47 · cited 5×
“[17] discuss how writers evaluate the suggestions provided and integrate them into diferent cognitive writing processes.”
BackgroundInfluentialSelf-citationThe inviscid text entry rate and its application as a grand goal for mobile text entry
Kristensson, P. O., & Vertanen, K. (2014) · Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services · doi:10.1145/2628363.2628405 · Cited by 39 · cited 1×
“Historically, HCI research for text entry has predominantly focused on efciency [68].”
BackgroundThe Limits of Expert Text Entry Speed on Mobile Keyboards with Autocorrect
Banovic, N., Sethapakdi, T., Hari, Y., Dey, A. K., & Mankoff, J. (2019) · Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/3338286.3340126 · Cited by 34
Sentiment Bias in Predictive Text Recommendations Results in Biased Writing
Arnold, K., Chauncey, K., & Gajos, K. (2018) · Canadian Human-Computer Communications Society / Société canadienne du dialogue humain-machine · doi:10.20380/gi2018.07 · Cited by 31
Perception vs. reality: measuring machine translation post-editing productivity
Gaspari, F., Toral, A., Naskar, S. K., et al. (2014) · Cited by 27
Lehmann, F., Markert, N., Dang, H., & Buschek, D. (2022) · Mensch und Computer 2022 · doi:10.1145/3543758.3543947 · Cited by 25 · cited 1×
“When the editor client received a text suggestion from the server, it revealed the suggestion letter by letter with random delays calibrated to resemble a co-writing process (cf. [71]).”
MethodSelf-citationJustCorrect: Intelligent Post Hoc Text Correction Techniques on Smartphones
Cui, W., Zhu, S., Zhang, M. R., Schwartz, H. A., Wobbrock, J. O., & Bi, X. (2020) · Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3379337.3415857 · Cited by 24
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
Huang, P.-S., Zhang, H., Jiang, R., Stanforth, R., Welbl, J., Rae, J. W., Maini, V., Yogatama, D., & Kohli, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03064 · Cited by 22
The Spread of Obesity in a Large Social Network over 32 Years
Stockman, J. A. (2009) · Yearbook of Pediatrics · doi:10.1016/s0084-3954(08)79134-6 · Cited by 16 · cited 1×
“On a societal level, researchers have shown that social infuence drives speculative markets [93], afects voting patterns [69], and contributes to the spread of unhealthy behaviors such as smoking and obesity [30, 31].”
BackgroundWhat If an AI Told You That 2 + 2 Is 5? Conformity to Algorithmic Recommendations.
Liel, Y., & Zalmanson, L. (2020) · AIS Electronic Library (AISeL) · Cited by 16
Self-citationWan, C.-S., & Chiou, W.-B. (2010) · Computers & Education · doi:10.1016/j.compedu.2009.07.016 · Cited by 15 · cited 1×
“Even participants with preformed opinions on the topic may have changed their attitudes by being encouraged to communicate a belief that runs counter to their own belief [12, 99].”
BackgroundInfluencing individually: fusing personalization and persuasion
Berkovsky, S., Freyne, J., & B, H. O.-K. A. (2015) · International Conference on Artificial Intelligence · Cited by 14
Does prediction really help in Marathi text input? empirical analysis of a longitudinal study
Dalvi, G., Ahire, S., Emmadi, N., Joshi, M., Joshi, A. N., Ghosh, S. K., Ghone, P., & Parmar, N. (2016) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/2935334.2935366 · Cited by 14
(2012) · Cited by 6
Layton, L. (2020) · Routledge eBooks · doi:10.4324/9781003023098-16 · Cited by 5
Spinning Sequence-to-Sequence Models with Meta-BackdoorsPreprint
Bagdasaryan, E., & Shmatikov, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.10443 · Cited by 4
Exposure to Opposing Views can Increase Political Polarization
Bail, C. A. (2018) · Harvard Dataverse · doi:10.7910/dvn/nseseh · Cited by 3
Persuasive Natural Language Generation -- A Literature Review
Duerr, S., & Gloor, P. A. (2021) · arXiv · doi:10.48550/arxiv.2101.05786 · cited 1×
“persuader uses language models to automate and optimize the production of content for advertising [39, 61] or misinformation [25, 67, 106].”
Background(2021)
Self-citationUniversal Language Model Fine-tuning for Text Classification
Howard, J., & Ruder, S. (2018) · arXiv · doi:10.48550/arxiv.1801.06146
Human heuristics for AI-generated language are flawed
Jakesch, M., Hancock, J., & Naaman, M. (2022) · arXiv · doi:10.48550/arxiv.2206.07271
Self-citationBeyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries
Dang, H., Benharrak, K., Lehmann, F., et al. (2022) · arXiv · doi:10.48550/arxiv.2208.09323
How Different Groups Prioritize Ethical Values for Responsible AI
Jakesch, M., Buçinca, Z., Amershi, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.07722
Self-citationShiller, R. J. (2015) · doi:10.1515/9781400865536
Simons, H. W. (2011) · doi:10.4324/9780203933039
Research works printed but not linked: 4
- [58] Daniel Kahneman. 2011. Thinking, fast and slow . Macmillan. -
- [81] D.G. Myers. 2008. Social Psychology . McGraw-Hill. -
- [90] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9. -
- [104] Daijin Yang, Yanpeng Zhou, Zhiyuan Zhang, Toby Jia-Jun Li, and LC Ray. 2022. AI as an Active Writer: Interaction Strategies with Generated Text in Human-AI Collaborative Fiction Writing 56-65. In IUI Workshops . -
Cited by
Generative artificial intelligence
Banh, L., & Strobel, G. (2023) · Electronic Markets · doi:10.1007/s12525-023-00680-1 · Cited by 579
“The promotion of explainable and responsible AI becomes paramount once more (Meske et al., 2022), especially in a technological state where distinguishing between generated and real content by GAI models is becoming increasingly more challenging (Jakesch et al., 2023b; Lehmann & Buschek, 2020).”
When combinations of humans and AI are useful: A systematic review and meta-analysis
Vaccaro, M., Almaatouq, A., & Malone, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02024-1 · Cited by 570
The Robots Are Here: Navigating the Generative AI Revolution in Computing Education
Prather, J., Denny, P., Leinonen, J., Becker, B. A., Albluwi, I., Craig, M., Keuning, H., Kiesler, N., Kohn, T., Luxton-Reilly, A., et al. (2023) · Proceedings of the 2023 Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3623762.3633499 · Cited by 396
More human than human: measuring ChatGPT political bias
Motoki, F., Pinho Neto, V., & Rodrigues, V. (2024) · Public Choice · doi:10.1007/s11127-023-01097-2 · Cited by 329
“Additionally, there is evidence that biased LLMs influence the views of users (Jakesch et al., 2023).”
InfluentialDraxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D., & Welsch, R. (2023) · arXiv · doi:10.48550/arxiv.2303.03283 · Cited by 284
Self-citation
Metadata
- Authors
- Jakesch, M., Bhat, A., Buschek, D., Zalmanson, L., & Naaman, M.
- Affiliations
- Cornell University (US)
- Indian Institute of Technology Bombay (IN)
- University of Bayreuth (DE)
- Tel Aviv University (IL)
- Cornell Tech (US)
- Venue
- Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2023-04-19
- Pages
- 1-15
- OpenAlex
- W4319167005
- Semantic Scholar
- 92999f7a304e866a2be7176e59c745481ed01042
- Cited by
- 265
- References
- 117
- Citations by year
- 2026: 77 · 2025: 85 · 2024: 85 · 2023: 18
- FWCI
- 66.30
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (CHS 1901151/1901329)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Opinion Dynamics and Social Influence
- Keywords
- large language models, AI writing tools, opinion formation, attitudes toward social media, user attitudes, GPT-3
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.