Examining Human-AI Collaboration for Co-Writing Constructive Comments Online
Open accessThe DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Abstract
This paper examines if large language models (LLMs) can help people write constructive comments on divisive social issues due to the difficulty of expressing constructive disagreement online. Through controlled experiments with 600 participants from India and the US, who reviewed and wrote constructive comments on threads related to Islamophobia and homophobia, we observed potential misalignment between how LLMs and humans perceive constructiveness in online comments. While the LLM was more likely to prioritize politeness and balance among contrasting viewpoints when evaluating constructiveness, participants emphasized logic and facts more than the LLM did. Despite these differences, participants rated both LLM-generated and human-AI co-written comments as significantly more constructive than those written independently by humans. Our analysis also revealed that LLM-generated comments integrated significantly more linguistic features of constructiveness compared to human-written comments. When participants used LLMs to refine their comments, the resulting comments were more constructive, more positive, less toxic, and retained the original intent. However, LLMs often distorted people's original views--especially when their stances were on a spectrum instead of being outright polarizing. Based on these findings, we discuss ethical and design considerations in using LLMs to facilitate constructive discourse online.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087 · cited 1×
“To analyze sentiment, we used VADER sentiment analysis tool as it is attuned to sentiments expressed in social media [40].”
Culture, dialectics, and reasoning about contradiction.
Peng, K., & Nisbett, R. E. (1999) · American Psychologist · doi:10.1037//0003-066x.54.9.741 · Cited by 1,755 · cited 8×
“Prior research shows that people from individualistic cultures (e.g., the US) prefer logical reasoning during argumentation [71] while those from collectivist cultures (e.g., India) prefer dialectical arguments during conflict [76].”
InfluentialProceedings of the Web Conference 2021
(2021) · doi:10.1145/3442381 · Cited by 1,488
Proceedings of the ACM on Human-Computer Interaction
Karahalios, K. G., Monroy‐Hernández, A., Lampinen, A., & Fitzpatrick, G. (2018) · Cited by 1,074
Facework competence in intercultural conflict: an updated face-negotiation theory
Ting-toomey, S., & Kurogi, A. (1998) · International Journal of Intercultural Relations · doi:10.1016/s0147-1767(98)00004-2 · Cited by 864 · cited 4×
“Linguistic Features.”
InfluentialProceedings of 48th Hawaii International Conference on System Sciences (HICSS)
Pöyry, E., Parvinen, P. M. T., & McFarland, R. (2015) · Cited by 643
Building Successful Online Communities: Evidence-Based Social Design
Kraut, R. E., Resnick, P., Kiesler, S. B., Ren, Y., Chen, Y., Burke, M., Kittur, N., Riedl, J., & Konstan, J. A. (2012) · Cited by 636
Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
(2025) · doi:10.1145/3706598 · Cited by 534
Universals of linguistic politeness
Hill, B., Ide, S., Ikuta, S., Kawasaki, A., & Ogino, T. (1986) · Journal of Pragmatics · doi:10.1016/0378-2166(86)90006-8 · Cited by 437
Investigating Cultural Alignment of Large Language Models
AlKhamissi, B., ElNokrashy, M., Alkhamissi, M., & Diab, M. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.671 · Cited by 379
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
(2001) · doi:10.1145/365024 · Cited by 350
Lee, M., Liang, P., & Yang, Q. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502030 · Cited by 345
Towards quality discourse in online news comments
Diakopoulos, N., & Naaman, M. (2011) · Proceedings of the ACM 2011 conference on Computer supported cooperative work · doi:10.1145/1958824.1958844 · Cited by 318 · cited 1×
“In doing so, we specifically focus on “constructiveness” , as online platforms routinely moderate and rank users’ comments based on constructiveness [25, 26] and often explicitly guide users to be constructive [62].”
Self-citationAI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances
Agarwal, D., Naaman, M., & Vashistha, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713564 · Cited by 306
Self-citationHoltgraves, T., & Joong-nam, Y. (1990) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.59.4.719 · Cited by 305 · cited 1×
“They tend to be more polite, indirect, and express their opinions in a moderate and compromising way than Americans during conflicts [37, 39, 71, 76, 91, 95].”
Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
Sorensen, T., Jiang, L., Hwang, J. D., Levine, S., Pyatkin, V., West, P., Dziri, N., Lu, X., Rao, K., Bhagavatula, C., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i18.29970 · Cited by 267
Co-Writing with Opinionated Language Models Affects Users’ Views
Jakesch, M., Bhat, A., Buschek, D., Zalmanson, L., & Naaman, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581196 · Cited by 265 · cited 1×
“The distinction between prompting LLMs to rephrase comments versus co-writing comments with auto-complete LLM suggestions holds significant design implications, as several studies show that auto-complete suggestions from LLMs shift people’s viewpoints and homogenize their writing [1, 41, 101].”
Self-citationA Systematic Review of Online Deliberation Research: A Review of Online Deliberation Research
Friess, D., & Eilders, C. (2015) · Policy & Internet · doi:10.1002/poi3.95 · Cited by 240 · cited 1×
“For example, Friess and Eilders [32] articulated constructive deliberation as finding common ground and providing new solutions.”
Constructive Conflict Resolution: Principles, Training, and Research
Deutsch, M. (1994) · Journal of Social Issues · doi:10.1111/j.1540-4560.1994.tb02395.x · Cited by 239 · cited 1×
“In cases of disagreement, constructive conflict resolution strategies involve showing cooperative attitudes and trust with conflicting parties [4, 24].”
Supporting reflective public thought with considerit
Kriplean, T., Morgan, J., Freelon, D., Borning, A., & Bennett, L. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145249 · Cited by 224 · cited 1×
“At the thread level, surfacing high-quality comments [6, 97], signaling existing tension within the conversation [12], displaying key points from other people’s comments [57], and pros and cons of different views expressed in the thread [56] have been shown to improve follow-up discussions.”
Mapping and Measuring Deliberation: Towards a New Deliberative Quality
Bächtiger, A., & Parkinson, J. (2019) · doi:10.1093/oso/9780199672196.001.0001 · Cited by 201
Kapoor, S., Hughes, P. C., Baldwin, J. R., & Blue, J. (2003) · International Journal of Intercultural Relations · doi:10.1016/j.ijintrel.2003.08.002 · Cited by 183
Ohbuchi, K., & Takahashi, Y. (1994) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.1994.tb01553.x · Cited by 172
Mcalister, S., Ravenscroft, A., & Scanlon, E. (2004) · Journal of Computer Assisted Learning · doi:10.1111/j.1365-2729.2004.00086.x · Cited by 167 · cited 1×
“Research shows that people often prefer pre-written questions [35] and sentence openers [64] that help them challenge others constructively instead of having to write everything from scratch.”
Design Matters! An Empirical Analysis of Online Deliberation on Different News Platforms
Esau, K., Frieß, D., & Eilders, C. (2017) · Policy & Internet · doi:10.1002/poi3.154 · Cited by 165 · cited 2×
“In response, HCI researchers have explored proactive strategies to promote healthy discourse, such as using social cues to highlight positive behavior [42, 77], and introducing frictions [47, 63, 81] and nudges [8, 28, 74, 93] to discourage people from using offensive language.”
Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study
Cao, Y., Zhou, L., Lee, S., Cabello, L., Chen, M., & Hershcovich, D. (2023) · Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP) · doi:10.18653/v1/2023.c3nlp-1.7 · Cited by 163
Argyle, L. P., Bail, C. A., Busby, E. C., Gubler, J. R., Howe, T., Rytting, C., Sorensen, T., & Wingate, D. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2311627120 · Cited by 144
Collier, B., & Bear, J. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145265 · Cited by 114
Sleeper, M., Cranshaw, J., Kelley, P. G., Ur, B., Acquisti, A., Cranor, L. F., & Sadeh, N. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466448 · Cited by 113
Liu, L. (2005) · Journal of Second Language Writing · doi:10.1016/j.jslw.2004.11.001 · Cited by 108
Zhang, Z., Gao, J., Dhaliwal, R. S., & Li, T. J.-J. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606800 · Cited by 108
Fong, C. J., Warner, J. R., Williams, K. M., Schallert, D. L., Chen, L., Williamson, Z. H., & Lin, S. (2016) · Learning and Individual Differences · doi:10.1016/j.lindif.2016.05.019 · Cited by 107 · cited 1×
“In case of student evaluation, peer reviews, and product reviews, constructive criticism is defined as those that are delivered respectfully and provide actionable feedback for improvement [31, 100, 105].”
Normative influences on thoughtful online participation
Sukumaran, A., Vezich, S., McHugh, M., & Nass, C. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979450 · Cited by 105 · cited 6×
“They are usually longer in length and take longer to write [52, 90].”
InfluentialWu, S.-Y., & Rubin, D. L. (2000) · Research in the Teaching of English · doi:10.58680/rte20001715 · Cited by 102
AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances
Agarwal, D., Naaman, M., & Vashistha, A. (2024) · arXiv · doi:10.48550/arxiv.2409.11360 · Cited by 99
Self-citationReconsidering Tweets: Intervening during Tweet Creation Decreases Offensive Content
Katsaros, M., Yang, K., & Fratamico, L. (2022) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v16i1.19308 · Cited by 87
Dang, H., Goller, S., Lehmann, F., & Buschek, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580969 · Cited by 83
Text structure of Korean speakers’ argumentative essays in English
Choi, Y. H. (1988) · World Englishes · doi:10.1111/j.1467-971x.1988.tb00226.x · Cited by 71
Seering, J., Fang, T., Damasco, L., Chen, M. '., Sun, L., & Kaufman, G. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300836 · Cited by 71 · cited 2×
“Moreover, giving people positive stimuli [81], nudging them to reflect before posting their comments [47], and making people wait before letting them post comments [63] also foster thoughtful and prosocial discussions.”
Reason against the machine? Future directions for mass online deliberation
Shortall, R., Itten, A. V., van der Meer, M., Murukannaiah, P. K., & Jonker, C. M. (2022) · Frontiers in Political Science · doi:10.3389/fpos.2022.946589 · Cited by 67 · cited 1×
“Moreover, existing studies on LLM-based interventions predominantly focus on Western populations, leaving a significant gap in understanding how to facilitate constructive discourse in different cultures [86].”
Shahid, F., & Vashistha, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581538 · Cited by 62 · cited 1×
“Close friends often use curse words to reflect real-life rapport, yet algorithms might flag and remove these interactions as inappropriate [85].”
Self-citationKim, J., Suh, S., Chilton, L. B., & Xia, H. (2023) · Proceedings of the 2023 ACM Designing Interactive Systems Conference · doi:10.1145/3563657.3595996 · Cited by 61
The politeness Package: Detecting Politeness in Natural Language
Yeomans, M., Kantor, A., & Tingley, D. (2019) · The R Journal · doi:10.32614/rj-2018-079 · Cited by 54
Croucher, S. M., Braziunaite, R., Homsey, D. M., Pillai, G., Saxena, J., Saldanha, A., Joshi, V., Jafri, I. A., Choudhary, P., Bose, L., et al. (2009) · Journal of Intercultural Communication Research · doi:10.1080/17475759.2009.505057 · Cited by 50
The Ghost in the Machine has an American accent: value conflict in GPT-3
Johnson, R. L., Pistilli, G., Menédez-González, N., Duran, L. D. D., Panai, E., Kalpokiene, J., & Bertulfo, D. J. (2022) · arXiv · doi:10.48550/arxiv.2203.07785 · Cited by 50 · cited 2×
“This is particularly important because not only a majority of users of online platforms are in non-Western regions, but also because LLMs are known to prioritize Western norms and values [9, 43] and homogenize writings towards Western styles [1].”
Hample, D., & Anagondahalli, D. (2015) · Journal of Intercultural Communication Research · doi:10.1080/17475759.2014.1000939 · Cited by 49
Someone Is Wrong on the Internet: Having Hard Conversations in Online Spaces
Baughan, A., Petelka, J., Yoo, C. J., Lo, J. L., Wang, S., Paramasivam, A., Zhou, A., & Hiniker, A. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449230 · Cited by 39
Xie, Y., Hample, D., & Wang, X. (2015) · Argumentation · doi:10.1007/s10503-015-9352-8 · Cited by 35
The Dearth of the Author in AI-Supported Writing
Kreminski, M. (2024) · Workshop on Intelligent and Interactive Writing Assistants (In2Writing) · doi:10.1145/3690712.3690725 · Cited by 35 · cited 1×
“Recent studies have found that LLM-assisted writing tools homogenize outputs, i.e., different users relying on the same LLM may produce more similar writing than they would without such assistance [55].”
Phutane, M., Seelam, A., & Vashistha, A. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732128 · Cited by 35
Self-citationWarner, M., Strohmayer, A., Higgs, M., & Coventry, L. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103468 · Cited by 31
Yan, L.-P., & Wang, X.-C. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.01.009 · Cited by 30
The art of peer review: Guidelines to become a credible and constructive peer reviewer
Weaver, M. L., Sundland, R. M., Adams, A. M., Faria, I. M. D. F., Feldman, H. A., Gudmundsdottir, H., Marmor, H. N., Miles, V., Ochoa, B. V., Ruff, S. M., et al. (2022) · Seminars in Vascular Surgery · doi:10.1053/j.semvascsurg.2022.10.002 · Cited by 24
Bias in AI Autocomplete Suggestions Leads to Attitude Shift on Societal IssuesPreprint
Williams‐Ceci, S., Jakesch, M., Bhat, A., Kadoma, K., Zalmanson, L., & Naaman, M. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/mhjn6 · Cited by 23
Self-citationAI Within Online Discussions: Rational, Civil, Privileged?
Carstens, J. A., & Frieß, D. (2024) · Minds and Machines · doi:10.1007/s11023-024-09658-0 · Cited by 23
Does Writing with Language Models Reduce Content Diversity?
Padmakumar, V., & He, H. (2023) · arXiv · doi:10.48550/arxiv.2309.05196 · Cited by 21
Promoting Online Civility Through Platform Architecture
Kim, J., McDonald, C., Meosky, P., Katsaros, M., & Tyler, T. R. (2022) · Journal of Online Trust and Safety · doi:10.54501/jots.v1i4.54 · Cited by 19
Forms of written arguments: A comparison between Japan and the United States
Suzuki, S. (2010) · International Journal of Intercultural Relations · doi:10.1016/j.ijintrel.2010.02.005 · Cited by 13
Park, I., Lee, D., & Sah, Y. J. (2022) · Behaviour and Information Technology · doi:10.1080/0144929x.2022.2114937 · Cited by 13 · cited 3×
“We calculated readability scores using SMOG index ( textstat library in Python) as reported in Park et al. [74].”
“Oops...”: Mobile Message Deletion in Conversation Error and Regret Remediation
Warner, M., Lascău, L., Cox, A. L., Brumby, D. P., & BLANDFORD, A. E. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445118 · Cited by 11
User-centred quality of UI interventions aiming to influence online news commenting behaviour
Kiskola, J., Olsson, T., Rantasila, A., Syrjämäki, A. H., Ilves, M., Isokoski, P., & Surakka, V. (2022) · Behaviour and Information Technology · doi:10.1080/0144929x.2022.2108723 · Cited by 9 · cited 2×
“…[2], cultural prompting [92], self-pluralism [104], modular pluralism [29], or value-pluralistic design [88]—which have been shown to increase cultural alignment in LLM-generated responses—are necessary to design HAI-collaborative systems to support prosocial discourse online on a global scale.”
Promoting Constructive Deliberation: Reframing for Receptiveness
Kambhatla, G., Lease, M., & Rajadesingan, A. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.294 · Cited by 9
Measuring Causal Effects of Civil Communication without Randomization
Liu, T., Ungar, L., Kording, K. P., & McGuire, M. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31365 · Cited by 7
Research works printed but not linked: 44
- George Berry and Sean J. Taylor. 2017. Discussion Quality Diffuses in the Digital Public Square. In Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1371-1380.
- Emilie Bossens, Elias Storms, and David Geerts. 2021. Improving the Debate: Interface Elements that Enhance Civility and Relevance in Online News Comments. In IFIP Conference on Human-Computer Interaction. Springer, Switzerland, 433-450.
- Jonathan P. Chang, Charlotte Schluger, and Cristian Danescu-Niculescu-Mizil. 2022. Thread With Caution: Proactively Helping Users Assess and Deescalate Tension in Their Online Discussions. Proc. ACM Hum.-Comput. Interact., Vol. 6, CSCW2, Article 545 (nov 2022), 37 pages.
- Incheol Choi, Minkyung Koo, and Jong An Choi. 2007. Individual differences in analytic versus holistic thinking. Personality and social psychology bulletin, Vol. 33, 5 (2007), 691-705.
- Christine Linda Cook, Juliette Schaafsma, Marjolijn L Antheunis, Suleman Shahid, Jih-Hsuan Tammy Lin, and Hanne W Nijtmans. 2021. Trolls without borders: A cross-cultural examination of victim reactions to verbal and silent aggression online. Frontiers in psychology, Vol. 12 (2021), 549955.
- Stephanie Cutler Yu Xia and Kacey Beddoes. 2022. A Growth Mindset for Peer Review: Guidelines for writing constructive peer reviews.
- Debarati Das, Karin De Langis, Anna Martin, Jaehyung Kim, Minhwa Lee, Zae Myung Kim, Shirley Hayati, Risako Owan, Bin Hu, Ritik Parkar, et al., 2024. Under the surface: Tracking the artifactuality of llm-generated data.
- Christine De Kock and Andreas Vlachos. 2021. I Beg to Differ: A study of constructive disagreement in online conversations. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, Paola Merlo, Jorg Tiedemann, and Reut Tsarfaty (Eds.). Association for Computational Linguistics, Online, 2017-2027.
- Nicholas Diakopoulos. 2015. Picking the NYT picks: Editorial criteria and automation in the curation of online news comments. ISOJ Journal, Vol. 6, 1 (2015), 147-166.
- Xiaohan Ding, Kaike Ping, Uma Sushmitha Gunturi, Buse Carik, Sophia Stil, Lance T Wilhelm, Taufiq Daryanto, James Hawdon, Sang Won Lee, and Eugenia H Rho. 2024. CounterQuill: Investigating the Potential of Human-AI Collaboration in Online Counterspeech Writing.
- Shangbin Feng, Taylor Sorensen, Yuhan Liu, Jillian Fisher, Chan Young Park, Yejin Choi, and Yulia Tsvetkov. 2024. Modular pluralism: Pluralistic alignment via multi-llm collaboration.
- Catherine Grevet, Loren G. Terveen, and Eric Gilbert. 2014. Managing political differences in social media. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, New York, USA, 1400-1408.
- Selin Gurgun, Emily Arden-Close, John McAlaney, Keith Phalp, and Raian Ali. 2023. Can We Re-design Social Media to Persuade People to Challenge Misinformation? An Exploratory Study. In International Conference on Persuasive Technology. Springer, Germany, 123-141.
- David Hitchcock and Bart Verheij. 2006. Arguing on the Toulmin model. Vol. 10. Springer, New York, USA.
- 10.1037/0022-3514.59.4.719 · doi:10.1037/0022-3514.59.4.719 · Paper page
- Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. 2017. SpaCy: Industrial-strength natural language processing in Python.
- Elise Karinshak, Sunny Xun Liu, Joon Sung Park, and Jeffrey T. Hancock. 2023. Working With AI to Persuade: Examining a Large Language Model's Ability to Generate Pro-Vaccination Messages. Proc. ACM Hum.-Comput. Interact., Vol. 7, CSCW1, Article 116 (2023), 29 pages.
- Whose Knowledge? Oxford Internet Institute and The Centre for Internet & Society (India). 2022. State of the Internet's Languages.
- Varada Kolhatkar and Maite Taboada. 2017a. Constructive Language in News Comments. In Proceedings of the First Workshop on Abusive Language Online, Zeerak Waseem, Wendy Hui Kyong Chung, Dirk Hovy, and Joel Tetreault (Eds.). ACL, Vancouver, Canada, 11-17.
- Varada Kolhatkar and Maite Taboada. 2017b. Using New York Times Picks to Identify Constructive Comments. In Proceedings of the 2017 EMNLP Workshop: Natural Language Processing meets Journalism, Octavian Popescu and Carlo Strapparava (Eds.). ACL, Copenhagen, Denmark, 100-105.
- 10.1145/3491102.3502030 · DOI · Paper page
- Yingqin Liu. 2009. The impact of cultural factors on Chinese and American college students' rhetorical choices in argumentative discourse: A contrastive study. Intercultural Communication Studies, Vol. 18, 1 (2009), 128.
- Teale W. Masrani, Jack Jamieson, Naomi Yamashita, and Helen Ai He. 2023. Slowing it Down: Towards Facilitating Interpersonal Mindfulness in Online Polarizing Conversations Over Social Media. Proc. ACM Hum.-Comput. Interact., Vol. 7, CSCW1, Article 90 (2023), 27 pages.
- Jimin Mun, Emily Allaway, Akhila Yerukola, Laura Vianna, Sarah-Jane Leslie, and Maarten Sap. 2023. Beyond Denouncing Hate: Strategies for Countering Implied Biases and Stereotypes in Language. In Findings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 9759-9777.
- Jimin Mun, Cathy Buerger, Jenny T Liang, Joshua Garland, and Maarten Sap. 2024. Counterspeakers' Perspectives: Unveiling Barriers and AI Needs in the Fight against Online Hate. In Proceedings of the CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24). ACM, New York, USA, Article 742, 22 pages.
- Courtney Napoles, Joel Tetreault, Aasish Pappu, Enrica Rosato, and Brian Provenzale. 2017. Finding Good Conversations Online: The Yahoo News Annotated Comments Corpus. In Proceedings of the 11th Linguistic Annotation Workshop, Nathan Schneider and Nianwen Xue (Eds.). Association for Computational Linguistics, Valencia, Spain, 11 pages.
- Vlad Niculae and Cristian Danescu-Niculescu-Mizil. 2016. Conversational Markers of Constructive Discussions. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Kevin Knight, Ani Nenkova, and Owen Rambow (Eds.). Association for Computational Linguistics, San Diego, California, 568-578.
- Richard E Nisbett, Kaiping Peng, Incheol Choi, and Ara Norenzayan. 2001. Culture and systems of thought: holistic versus analytic cognition. Psychological review, Vol. 108, 2 (2001), 291.
- Ara Norenzayan, Edward E Smith, Beom Jun Kim, and Richard E Nisbett. 2002. Cultural preferences for formal versus intuitive reasoning. Cognitive science, Vol. 26, 5 (2002), 653-684.
- Kaiping Peng. 1997. Naive dialecticism and its effects on reasoning and judgment about contradiction. Ph.D. Dissertation. University of Michigan.
- 10.1037/0003-066X.54.9.741 · doi:10.1037/0003-066x.54.9.741 · Paper page
- Ashwin Rajadesingan, Carolyn Duran, Paul Resnick, and Ceren Budak. 2021. 'Walking Into a Fire Hoping You Don't Catch': Strategies and Designs to Facilitate Cross-Partisan Online Discussions. Proceedings of the ACM on Human-Computer Interaction, Vol. 5, CSCW2 (2021), 1-30.
- Carlos Ruiz, David Domingo, Josep Lluís Micó, Javier Díaz-Noci, Koldo Meso, and Pere Masip. 2011. Public sphere 2.0? The democratic qualities of citizen debates in online newspapers. The International journal of press/politics, Vol. 16, 4 (2011), 463-487.
- Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. 2022. Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (Eds.). Association for Computational Linguistics, Seattle, United States, 5884-5906.
- Farhana Shahid, Dhruv Agarwal, and Aditya Vashistha. 2025. One Style Does Not Regulate All: Moderation Practices in Public and Private WhatsApp Groups. Proceedings of the ACM on Human-Computer Interaction, Vol. 9, CSCW2 (2025), 1-30.
- John Suler. 2004. The online disinhibition effect. Cyberpsychology & behavior, Vol. 7, 3 (2004), 321-326.
- Yan Tao, Olga Viberg, Ryan S Baker, and René F Kizilcec. 2024. Cultural bias and cultural alignment of large language models. PNAS nexus, Vol. 3, 9 (2024), 346.
- Samuel Hardman Taylor, Dominic DiFranzo, Yoon Hyung Choi, Shruti Sannon, and Natalya N. Bazarova. 2019. Accountability and Empathy by Design: Encouraging Bystander Intervention to Cyberbullying on Social Media. Proc. ACM Hum.-Comput. Interact., Vol. 3, CSCW, Article 118 (nov 2019), 26 pages.
- Michael Henry Tessler, Michiel A. Bakker, Daniel Jarrett, Hannah Sheahan, Martin J. Chadwick, Raphael Koster, Georgina Evans, Lucy Campbell-Gillingham, Tantum Collins, David C. Parkes, Matthew Botvinick, and Christopher Summerfield. 2024. AI can help humans find common ground in democratic deliberation. Science, Vol. 386, 6719 (2024), eadq2852.
- Stella Ting-Toomey, Ge Gao, Paula Trubisky, Zhizhong Yang, Hak Soo Kim, Sung-Ling Lin, and Tsukasa Nishida. 1991. Culture, face maintenance, and styles of handling interpersonal conflict: A study in five cultures. International Journal of conflict management, Vol. 2, 4 (1991), 275-296.
- Yixue Wang and Nicholas Diakopoulos. 2022. Highlighting High-quality Content as a Moderation Strategy: The Role of New York Times Picks in Comment Quality and Engagement. Trans. Soc. Comput., Vol. 4, 4, Article 13 (2022), 24 pages.
- Meng Xia, Qian Zhu, Xingbo Wang, Fei Nie, Huamin Qu, and Xiaojuan Ma. 2022. Persua: A Visual Interactive System to Enhance the Persuasiveness of Arguments in Online Discussion. Proc. ACM Hum.-Comput. Interact., Vol. 6, CSCW2, Article 319 (2022), 30 pages.
- Justine Zhang, Jonathan Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Dario Taraborelli, and Nithum Thain. 2018. Conversations Gone Awry: Detecting Early Signs of Conversational Failure. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Melbourne, Australia, 1350-1361.
- Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2019. BERTScore: Evaluating Text Generation with BERT. CoRR, Vol. abs/1904.09675 (2019), 43. · arXiv
Cited by
Where Do I 'Add the Egg'?: Exploring Agency and Ownership in AI Creative Co-Writing Systems
Carrera, D., Thomas-Mitchell, J., & Wigdor, D. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791952 · Cited by 3
“Many interfaces resembled commercial instantiations of this archetype, utilizing buttons, dropdown menus, and keywords (in creative writing [51, 58, 70, 80] and other writing contexts [28, 38, 63]).”
Xiao, X., Chen, H., Song, Q., & Yang, Y. (2025) · Science Communication · doi:10.1177/10755470251384848 · Cited by 1
Rauxloh, H. (2026) · Communication Methods and Measures · doi:10.1080/19312458.2026.2704557
Large language models for computational social science
Xu, F., Shao, C., Chen, L., Zeng, Q., Chen, Z., Sukiennik, N., Wang, J., Gao, C., Wang, H., Lian, J., et al. (2026) · Nature Human Behaviour · doi:10.1038/s41562-026-02539-9 · Cited by 1
Metadata
- Authors
- Shahid, F., Dittgen, M., Naaman, M., & Vashistha, A.
- Affiliations
- Cornell University (US)
- Cornell Tech (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2025-10-16
- Volume
- 9
- Issue
- 7
- Pages
- 1-33
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4415256795
- Cited by
- 3
- References
- 62
- Citations by year
- 2026: 2 · 2025: 1
- FWCI
- 1.21
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- human-AI collaboration, large language models, constructive disagreement, online comments, Islamophobia, homophobia, comment moderation, politeness, toxic language, AI-generated text
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.