Opportunities and Risks of LLMs for Scalable Deliberation with Polis

“…routing is rooted in the fields of Active Learning (Cohn et al., 1996), also known as Sequential Design of Experiment in statistics, Bayesian Optimisation using Gaussian Processes (Rasmussen, 2006), Exploration/Exploitation in Reinforcement Learning and Bandits Algorithms Sutton and Barto (1998).”

“Nevertheless, we continued to monitor advances in NLP, and recognized a critical threshold in capacity had been crossed with the advent of Large Language Models (LLMs), e.g. Brown et al. (2020).”

“By arranging the votes in a vote matrix and applying dimension reduction and clustering techniques—specifically Principle Components Analysis (PCA) (Pearson, 1901) and K-means clustering (MacQueen, 1967)—Polis is able to learn a 2-dimensional opinion space and opinion groups , which are used as the…”

“Furthermore, LLMs may reflect and amplify societal biases present in their training data (Bender et al., 2021a; Abid et al., 2021a; Cheng et al., 2023) – including political biases – which may affect what information they extract and emphasize when summarizing deliberative discourse (see Section…”

“…et al., 2020; Kurita et al., 2019; Basta et al., 2019; Abid et al., 2021b; Sap et al., 2020), prompt injection attacks (Greshake et al., 2023; Perez and Ribeiro, 2022), so called “hallucinations” (LLMs making things up when they don’t know something; Ji et al., 2023b), and basic transparency.”

“LLMs are artificial neural networks trained on massive corpora of training data (typically text) in order to predict new textual completions (OpenAI, 2023; Touvron et al., 2023; Bai et al., 2022; Schulman et al., 2022).”

“Moderation of hate speech remains central to ensuring the integrity of a Polis con-versation’s result, as violent and/or dignity-impairing speech limits full participation (Habermas, 1962, 1981).”

“…mechanisms, it may eventually be possible to use these models to formally quantify aspects of the summarization process in unprecedented ways, though currently there is debate about whether these mechanisms can be relied on for interpretability (Jain and Wallace, 2019; Wiegreffe and Pinter, 2019).”

“In the role of deliberative assistant, this is potentially problematic if we are relying on a model as a multi-partial (Harding, 1986) facilitation agent when it aligns more closely with one of the groups than the other.”

“…that there is content there to vote on for the very first tranche of participants, and set the tone of the conversation (not dissimilar to how Sequential Design of Experiments, Active Learning (Cohn et al., 1996), and Bayesian Optimisation (Rasmussen, 2006) are seeded with a set of initial points).”

“Prior work has proposed automated metrics to detect such errors (Maynez et al., 2020; Durmus et al., 2020; Kryscinski et al., 2020; Goyal and Durrett, 2021); however, they do not always correlate well with human judgements (Fabbri et al., 2021; Durmus et al., 2022).”

“…mechanisms, it may eventually be possible to use these models to formally quantify aspects of the summarization process in unprecedented ways, though currently there is debate about whether these mechanisms can be relied on for interpretability (Jain and Wallace, 2019; Wiegreffe and Pinter, 2019).”

“For these reasons, generative testing and property-based testing have emerged as solutions which involve stochastically generating data to more thoroughly explore the possible state space (Claessen and Hughes, 2000; Pacheco and Ernst, 2007).”

“Prior work has proposed automated metrics to detect such errors (Maynez et al., 2020; Durmus et al., 2020; Kryscinski et al., 2020; Goyal and Durrett, 2021); however, they do not always correlate well with human judgements (Fabbri et al., 2021; Durmus et al., 2022).”

“…et al., 2023; Hutchinson et al., 2020; Kurita et al., 2019; Basta et al., 2019; Abid et al., 2021b; Sap et al., 2020), prompt injection attacks (Greshake et al., 2023; Perez and Ribeiro, 2022), so called “hallucinations” (LLMs making things up when they don’t know something; Ji et al., 2023b),…”

“Prior work has proposed automated metrics to detect such errors (Maynez et al., 2020; Durmus et al., 2020; Kryscinski et al., 2020; Goyal and Durrett, 2021); however, they do not always correlate well with human judgements (Fabbri et al., 2021; Durmus et al., 2022).”

“A potential “third way” between strict and non-strict moderation might leverage LLMs (or other NLP technology) to place potentially toxic or hateful content in a queue for human review (Jahan and Oussalah, 2021; MacAvaney et al., 2019; Wei et al., 2021; Chiu et al., 2022).”

“These human judgments can be used to as a feedback to the models and better align the automated summarization with what participants feel captures the essence of the discourse, a sort of Deliberative Reinforcement Learning from Human Feedback (RLHF) (Ziegler et al., 2020; Lambert et al., 2022).”

“However, while LLM technology has massively broadened the scope of what’s possible to achieve with Natural Language Processing, there remain significant limitations and risks associated with their use (Ganguli et al., 2022b,a; Weidinger et al., 2021; Bom-masani et al., 2022; Bender et al., 2021b).”

“Moreover, many LLM Terms of Service (including those of OpenAI’s GPT and Anthropic’s Claude) require that people be informed when they are looking at model generated content.”

“For some domains, LLMs have already been shown to be effective at summarizing textual and numeric information, in some cases judged as on par with human written summaries (Zhang et al., 2023; Singh et al., 2023).”

“These human judgments can be used to as a feedback to the models and better align the automated summarization with what participants feel captures the essence of the discourse, a sort of Deliberative Reinforcement Learning from Human Feedback (RLHF) (Ziegler et al., 2020; Lambert et al., 2022).”

“…LLMs, there remain issues around bias (Santurkar et al., 2023; Hutchinson et al., 2020; Kurita et al., 2019; Basta et al., 2019; Abid et al., 2021b; Sap et al., 2020), prompt injection attacks (Greshake et al., 2023; Perez and Ribeiro, 2022), so called “hallucinations” (LLMs making things up when…”

“Furthermore, LLMs may reflect and amplify societal biases present in their training data (Bender et al., 2021a; Abid et al., 2021a; Cheng et al., 2023) – including political biases – which may affect what information they extract and emphasize when summarizing deliberative discourse (see Section…”

“Prior work has proposed automated metrics to detect such errors (Maynez et al., 2020; Durmus et al., 2020; Kryscinski et al., 2020; Goyal and Durrett, 2021); however, they do not always correlate well with human judgements (Fabbri et al., 2021; Durmus et al., 2022).”

“Despite significant efforts on the part of researchers developing LLMs, there remain issues around bias (Santurkar et al., 2023; Hutchinson et al., 2020; Kurita et al., 2019; Basta et al., 2019; Abid et al., 2021b; Sap et al., 2020), prompt injection attacks (Greshake et al.”

“Following Park et al. (2022), an entire Polis conversation could be simulated using an LLM-generated collection of participants—each with unique backgrounds, expertise, and ideological alignments—who comment and vote in response to a prompt.”

“Citizen assemblies facilitate intensive deliberation, at a considerable cost of time and money (Landemore, 2022).”

“Furthermore, LLMs may reflect and amplify societal biases present in their training data (Bender et al., 2021a; Abid et al., 2021a; Cheng et al., 2023) – including political biases – which may affect what information they extract and emphasize when summarizing deliberative discourse (see Section…”

“…on the part of researchers developing LLMs, there remain issues around bias (Santurkar et al., 2023; Hutchinson et al., 2020; Kurita et al., 2019; Basta et al., 2019; Abid et al., 2021b; Sap et al., 2020), prompt injection attacks (Greshake et al., 2023; Perez and Ribeiro, 2022), so called…”

“Prior work has found this type of participatory approach, where the users and stakeholders of an AI system are involved in evaluating and improving its outputs, can help identify errors, biases, and misrepresentations (Lee et al., 2022; Yang et al., 2018).”

“Surprisingly, we found that a plain LLM (Askell et al., 2021), without fine-tuning or RLHF training, was nearly perfectly statistically calibrated at predicting agreement, as seen in Figure 4.”

“For these reasons, generative testing and propertybased testing have emerged as solutions which involve stochastically generating data to more thoroughly explore the possible state space (Claessen and Hughes, 2000; Pacheco and Ernst, 2007).”

“Despite significant efforts on the part of researchers developing LLMs, there remain issues around bias (Santurkar et al., 2023; Hutchinson et al., 2020; Kurita et al., 2019; Basta et al., 2019; Abid et al., 2021b; Sap et al., 2020), prompt injection attacks (Greshake et al., 2023; Perez and…”

“…et al., 2020; Kurita et al., 2019; Basta et al., 2019; Abid et al., 2021b; Sap et al., 2020), prompt injection attacks (Greshake et al., 2023; Perez and Ribeiro, 2022), so called “hallucinations” (LLMs making things up when they don’t know something; Ji et al., 2023b), and basic transparency.”

“A potential “third way” between strict and non-strict moderation might leverage LLMs (or other NLP technology) to place potentially toxic or hateful content in a queue for human review (Jahan and Oussalah, 2021; MacAvaney et al., 2019; Wei et al., 2021; Chiu et al., 2022).”

“Prior work has proposed automated metrics to detect such errors (Maynez et al., 2020; Durmus et al., 2020; Kryscinski et al., 2020; Goyal and Durrett, 2021); however, they do not always correlate well with human judgements (Fabbri et al., 2021; Durmus et al., 2022).”

“Its methodology was also built on as a basis for Twitter’s Community Notes (Wojcik et al., 2022).”

“This reflects the pattern of recursive summarization explored with human summarizers in Zhang et al. (2017).”

“Prior work has found this type of participatory approach, where the users and stakeholders of an AI system are involved in evaluating and improving its outputs, can help identify errors, biases, and misrepresentations (Lee et al., 2022; Yang et al., 2018).”

“Prior work has proposed automated metrics to detect such errors (Maynez et al., 2020; Durmus et al., 2020; Kryscinski et al., 2020; Goyal and Durrett, 2021); however, they do not always correlate well with human judgements (Fabbri et al., 2021; Durmus et al., 2022).”

“The distribution of these values across comments are shown in 2023) have demonstrated evidence that other LLMs may trend left-leaning politically.”

“A potential “third way” between strict and non-strict moderation might leverage LLMs (or other NLP technology) to place potentially toxic or hateful content in a queue for human review (Jahan and Oussalah, 2021; MacAvaney et al., 2019; Wei et al., 2021; Chiu et al., 2022).”

“Experiments have already demonstrated that LLMs trained on media content can predict public opinion (Chu et al., 2023).”

“The Remesh platform (Bilich et al., 2019) has pioneered the use of LLMs for inferring opinion data in the absence of votes across and within conversations by integrating semantic similarity via text embeddings into its Latent Factor Model (Konya et al.”

“Despite significant efforts on the part of researchers developing LLMs, there remain issues around bias (Santurkar et al., 2023; Hutchinson et al., 2020; Kurita et al., 2019; Basta et al., 2019; Abid et al., 2021b; Sap et al., 2020), prompt injection attacks (Greshake et al., 2023; Perez and…”

“For some domains, LLMs have already been shown to be effective at summarizing textual and numeric information, in some cases judged as on par with human written summaries (Zhang et al., 2023; Singh et al., 2023).”

“Machine learning models reproduce various content biases inherent to their training data, including gender bias, racial or ethnic bias, bias for negative and threat-related information, and socio-economic bias 69,86–90 .”

“For example, an LLM could manage speaker queues (that is, who should speak to whom, when and on what topic) or request elaboration if a statement cannot be confidently classified with a previously seen topic label 111,112 .”

“This approach also has the potential to increase the risk of failing to discover novel insights, as the model might not adequately question or critique their initial findings 39 .”

“This can be done in a conversational manner, making it easier for individuals to voice their opinions or concerns and for public servants to better understand farmers’ pain points based on millions of queries [67].”

“They have successfully coordinated large groups for communication (Small et al., 2023) and could present an opportunity for more extensive collaborations.”

“Work has been done in applying AI and Large Language Models (LLMs) to deliberation-specific tasks such as consensus building [9], summarization/moderation on deliberation platforms [10], argument extraction [11,12], fact-checking [13], and misin-formation detection [14].”

“We expect our work to spark further interest in this area and promote future research that enables the widespread adoption of AI tools in the public sector that are thoughtfully deployed to improve democracy.”

“LLMs have been applied to aspects of civic decision-making, such as finding common ground in deliberation contexts [24, 97], group preference balancing [82], and controlling for differences in decision-makers’ language use and expertise [114].”

“Small et al. [205] also evaluate LLMs as consensus generators within a political deliberation platform, using LLMs to perform topic modeling across group dis/agreements.”

“While existing systems have explored AI support for specific engagement functions [92] and participatory contexts [75, 110], our work advances this space by demonstrating how narrative-based human-AI collaboration can operate effectively in representative political contexts where synthesis must…”

“Following the guidelines in Small et al. (2023) for generating diverse seed statements with LLMs, we use GPT 5.1-mini (OpenAI, 2025a) to generate 8 seed statements for each question with a 1-shot prompt.”