On Epistemic Diversity in Large Language Models

Classified as probably not a substantial claim.

Large language models (LLMs) are increasingly used as knowledge tools for answering questions, explaining concepts, drafting arguments, and supporting inquiry.

“Large language models (LLMs) are increasingly used not only as predictive tools, but as knowledge tools for answering questions, explaining concepts, drafting arguments, and supporting inquiry (Chatterji et al., 2025).”

• cites Chatterji et al., 2025 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Jain et al. (2025) developed task-specific methods to mitigate output homogeneity.

“Concurrent work by Jain et al. (2025) develops task-specific methods to mitigate output homogeneity.”

• cites Jain et al. (2025) sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Jury Theorems argue that under certain conditions, a diverse set of users can arrive at better solutions than experts.

“Jury Theorems argue that under certain conditions, a diverse set of users can arrive at better solutions than experts (Hong & Page, 2004;Dietrich & Spiekermann, 2021).”

• cites Hong & Page, 2004 sourceSupported
• cites Dietrich & Spiekermann, 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Guo et al. (2025) showed that model outputs lack the richness of human text across lexical, syntactic, and semantic dimensions.

“Guo et al. (2025) analyze linguistic diversity across lexical, syntactic, and semantic dimensions, showing that model outputs lack the richness of human text.”

• cites Guo et al. (2025) sourcePartly supported

  The “Model Output Diversity” passage in Section 5.2 reports that LLMs lag behind humans in lexical, syntactic, and semantic diversity for story generation. The claim omits that task qualification: the same passage says models exceed human references in lexical and semantic diversity for next-utterance generation. (LLM, conf=0.90)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

LLM-as-a-judge evaluation is known to be unreliable for expert-domain quality judgment.

“LLM-as-a-judge evaluation is known to be unreliable for expert-domain quality judgment (Dorner et al., 2025;Szymanski et al., 2025).”

• cites Dorner et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Szymanski et al., 2025 sourcePartly supported

  The paper reports that LLM judges agreed with subject-matter experts 68% of the time in dietetics and 64% in mental health (p. 1). Its Discussion (p. 13) warns against relying solely on LLMs for expert-knowledge tasks. These findings support a concern about reliability, but the claim states it categorically across expert domains, while the study examines two domains and describes the broader validity question as uncertain. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim. May combine two claims.

Recent work highlights risks of homogenization and loss of minority viewpoints.

“Recent work highlights risks of homogenization and loss of minority viewpoints (Fazelpour & Fleisher, 2025).”

• cites Fazelpour & Fleisher, 2025 sourceSupported

Classified as probably not a substantial claim.

Jiang et al. (2025) analyzed open-ended queries from WildChat and showed that models produce semantically similar outputs.

“Jiang et al. (2025) analyze open-ended queries from WildChat and show that models produce semantically similar outputs.”

• cites Jiang et al. (2025) sourceSupported

May combine two claims.

Stronger probing can recover additional valid responses, showing that some plurality is present latently even when it is not readily surfaced by default.

“Stronger probing can recover additional valid responses, showing that some plurality is present latently even when it is not readily surfaced by default.”

May combine two claims.

Stronger prompting can recover some additional diversity, but a substantial gap remains.

“While stronger prompting can recover some additional diversity, a substantial gap remains.”

May combine two claims.

Under iterative prompting, 22% of rounds fail to add a new valid name, mostly because the model repeats an already-excluded answer.

“Under iterative prompting, 22% of rounds fail to add a new valid name, mostly because the model repeats an already-excluded answer, and some models produce substantial numbers of invalid names (71% for Qwen 2.5).”

May combine two claims.

Concentration persists: models still favor highly salient individuals and leave much of the answer space uncovered.

“Concentration also persists: models still favor highly salient individuals and leave much of the answer space uncovered (Figure 11).”

May combine two claims.

At a matched budget of ten responses, all three diversity-seeking protocols recover more unique valid names than accessible sampling for every model tested, with median gains of +85, +83, and +51 names in total across prompts.

“At a matched budget of ten responses, all three diversity-seeking protocols recover more unique valid names than accessible sampling for every model tested, with median gains of +85, +83, and +51 names in total across prompts.”

May combine two claims.

When asked to write about a famous physicist, models named only about 6% of the individuals listed on the corresponding Wikipedia page, concentrated on a handful of individuals, most often Einstein.

“Likewise, when asked to write about a famous physicist, models named only about 6% of the individuals listed on the corresponding Wikipedia page, concentrated on a handful of individuals, most often Einstein.”

May combine two claims.

In the professions domain, models repeatedly concentrate on a small set of canonical individuals; in the proofs domain, they often return the same proof strategy despite the existence of accepted alternatives.

“In the professions domain, models repeatedly concentrate on a small set of canonical individuals; in the proofs domain, they often return the same proof strategy despite the existence of accepted alternatives.”

May combine two claims.

In the multi-turn protocol, models recover 8.2 unique valid names on average, compared to 2.9 under accessible sampling and 7.8 under latent probing, averaged over the 36 prompts and 10 models.

“In the multi-turn protocol, models recover 8.2 unique valid names on average, compared to 2.9 under accessible sampling and 7.8 under latent probing (averaged over the 36 prompts and 10 models).”

The quote only partly states this claim.

Qwen 2.5 and Qwen 3 often produce hallucinated entries under iterative prompting.

“For example, Gemini 3 produces on average 88 names, Llama 3.1 69, and Qwen 2.5 and Qwen 3 only 26 and 31, often with hallucinated entries (Table 1).”

• AssumptionRests on an assumption: The Wikipedia lists of well-known practitioners provide a valid and sufficiently complete ground truth for the set of valid names in the professions dataset.

May combine two claims.

Multi-turn and latent probing show no detectable difference (p = 0.70; bootstrap 95% CI on the mean paired difference [-6.7, +21.0] names), while the multi-output protocol recovers fewer unique names per model than either.

“Multi-turn and latent probing show no detectable difference (p = 0.70; bootstrap 95% CI on the mean paired difference [-6.7, +21.0] names), while the multi-output protocol recovers fewer unique names per model than either.”

May combine two claims.

While Gemini 3 achieves the highest performance, other models fail to cover much of the valid answer space (e.g., Qwen 2.5, Qwen 3).

“While Gemini 3 achieves the highest performance, other models fail to cover much of the valid answer space (e.g.,Qwen 2.5, Qwen 3).”

May combine two claims.

Pooled across models, the multi-output protocol yields more distinct valid names in total (345) than latent probing (275), and fewer than multi-turn prompting (511), against 227 under accessible sampling.

“Pooled across models, however, it yields more distinct valid names in total (345) than latent probing (275), and fewer than multi-turn prompting (511), against 227 under accessible sampling.”

May combine two claims.

Higher temperature changes the framing and prose of the generated text, but not the individual it is about.

“Example outputs illustrate why: higher temperature changes the framing and prose of the generated text, but not the individual it is about (Table 4).”

May combine two claims.

Qwen models and Mistral show the highest invalid rates, with 71% of names judged invalid for the profession for Qwen 2.5.

“Qwen models and Mistral show the highest invalid rates, with 71% of names judged invalid for the profession for Qwen 2.5.”

May combine two claims.

The number of valid answers varies widely, with some questions having multiple but countable answers (n = 288), and others admitting infinitely many (n = 89).

“We also observe that the number of valid answers varies widely, with some questions having multiple but countable answers (n = 288), and others admitting infinitely many (n = 89).”

May combine two claims.

Even when users ask for alternatives, models continue to favor a subset of highly salient individuals, and some parts of the reference space remain uncovered.

“Even when users ask for alternatives, models continue to favor a subset of highly salient individuals, and some parts of the reference space remain uncovered.”

May combine two claims.

Across all models, 78% of rounds successfully add a new name to the exclusion list, while 22% stall, almost entirely because the model re-extracts someone already excluded (17%) or the extractor returns no person (5%).

“Across all models, 78% of rounds successfully add a new name to the exclusion list, while 22% stall, almost entirely because the model re-extracts someone already excluded (17%) or the extractor returns no person (5%).”

May combine two claims.

Latent diversity is higher but still constrained.

“Latent diversity is higher but still constrained.”

• AssumptionRests on an assumption: The Wikipedia lists of well-known practitioners provide a valid and sufficiently complete ground truth for the set of valid names in the professions dataset.

May combine two claims.

Under iterative prompting, models produce additional valid names but still fail to cover the full answer space.

“Under iterative prompting, models produce additional valid names but still fail to cover the full answer space (Figure 4b and Figure 7 for per-profession curves).”

• AssumptionRests on an assumption: The Wikipedia lists of well-known practitioners provide a valid and sufficiently complete ground truth for the set of valid names in the professions dataset.

May combine two claims.

Varying temperature has little effect on this behavior, indicating that stochasticity alone does not meaningfully expand the explored answer space.

“Varying temperature has little effect on this behavior, indicating that stochasticity alone does not meaningfully expand the explored answer space (Figure 10).”

The quote only partly states this claim.

Small phrasing variations of the input prompts yield only small differences: 32.7 unique valid names on average (per model-protocol setting) for the 'any' phrasing, compared to 33.2 for the original 'a' prompt.

“We only find small differences, yielding 32.7 unique valid names on average (per model-protocol setting) for the”

May combine two claims.

Stagnation rises sharply with round index and is worst for Qwen 3 (42%) and best for Gemini 3 (5%), with duplicate repeats making up ∼ 80-97% of stalled rounds for strong models like GPT-4o and Gemini 3.

“Stagnation rises sharply with round index and is worst for Qwen 3 (42%) and best for Gemini 3 (5%), with duplicate repeats making up ∼ 80-97% of stalled rounds for strong models like GPT-4o and Gemini 3.”

May combine two claims.

In the professions dataset, notable individuals such as Hans Zimmer and John Williams are never mentioned, and across all models only 2-24% of reference names are covered.

“In the professions dataset, notable individuals such as Hans Zimmer and John Williams are never mentioned, and across all models only 2-24% of reference names are covered (Table 3).”

May combine two claims.

For a substantial fraction of queries, one individual accounts for the majority of all responses, with Llama 4 and Gemini 3 the most concentrated.

“Figure 5a shows the maximum share of responses assigned to a single individual for each query: for a substantial fraction of queries, one individual accounts for the majority of all responses, with Llama 4 and Gemini 3 the most concentrated.”

May combine two claims.

The most common failure modes we identify are repeated valid answers, repeated invalid answers and hallucinated persons, no extractable entities, and citing real persons outside of the target profession.

“The most common failure modes we identify are repeated valid answers, repeated invalid answers and hallucinated persons, no extractable entities, and citing real persons outside of the target profession.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The two notions (group and epistemic diversity) can overlap, for example when knowledge correlates with group membership.

“The two notions can overlap, for example when knowledge correlates with group membership, as in feminist epistemology or religious values.”

Probably not a substantial claim: may not pass S2 (checkable).

In open-ended domains, measurement may require grouping responses into types, strategies, or conceptual clusters rather than literally enumerating all possibilities.

“In open-ended domains, measurement may require grouping responses into types, strategies, or conceptual clusters rather than literally enumerating all possibilities.”

Probably not a substantial claim: may not pass S4 (based).

A system may be accurate and yet still be epistemically narrow if it repeatedly presents only a few canonical routes despite multiple valid answers.

“A system may be accurate and yet still be epistemically narrow if, in contexts where alternatives would be useful, it repeatedly presents only a few canonical routes despite the existence of multiple valid answers.”

Classified as probably not a substantial claim.

Heterogeneity in viewpoints helps guard against collective blind spots.

“This is a core epistemic benefit of diversity: heterogeneity in viewpoints helps guard against collective blind spots.”

Classified as probably not a substantial claim.

Social epistemologists have argued that heterogeneous communities can reveal blind spots and correct distortions that no isolated reasoner can easily detect.

“Social epistemologists have argued that heterogeneous communities can reveal blind spots and correct distortions that no isolated reasoner can easily detect (Longino, 1990).”

• cites Longino, 1990Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

Diverse viewpoints and methods can make knowledge production more robust and less biased.

“Diverse viewpoints and methods can make knowledge production more robust and less biased (Longino, 1990;Kellert et al., 2006).”

• cites Longino, 1990Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Kellert et al., 2006Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Epistemic narrowness is not only a property of the model, but also of the model-interface interaction through which users encounter knowledge.

“This distinction matters because epistemic narrowness is not only a property of the model, but also of the model-interface interaction through which users encounter knowledge.”

May combine two claims.

Group diversity does not imply epistemic diversity; a system may represent multiple groups while surfacing only a narrow subset of knowledge.

“A system may represent multiple groups while still surfacing only a narrow subset of the knowledge available across them.”

Probably not a substantial claim: may not pass S4 (based).

Current approaches to diversity in AI largely focus on who is represented in model behavior.

“However, diversity encompasses a wide range of meanings, and current approaches largely focus on who is represented in model behavior.”

Classified as probably not a substantial claim.

Pluralist traditions in philosophy of science have emphasized that inquiry often advances through the coexistence of multiple models, explanations, or methods.

“Pluralist traditions in philosophy of science have emphasized that inquiry often advances through the coexistence of multiple models, explanations, or methods rather than through immediate convergence on a single privileged representation (Kellert et al., 2006;Feyerabend, 2002).”

• cites Kellert et al., 2006Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Feyerabend, 2002Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

Standpoint epistemology stresses that knowledge is situated, and that different positions can disclose different dimensions of a domain.

“Standpoint epistemology likewise stresses that knowledge is situated, and that different positions can disclose different dimensions of a domain (Harding, 1991;Collins, 2002).”

• cites Harding, 1991Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Collins, 2002Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S4 (based).

Epistemic diversity has not been systematically formalized in the context of LLMs.

“While epistemic diversity is an important aspect of knowledge generation, it has not been systematically formalized in the context of LLMs.”

Probably not a substantial claim: may not pass S2 (checkable).

In knowledge-intensive settings, the notion of epistemic diversity is applicable.

“In a knowledge-intensive setting, like writing about famous physicists or proving mathematical theorems, the notion of epistemic diversity is applicable.”

Probably not a substantial claim: may not pass S2 (checkable).

Epistemic diversity is conceptually distinct from group-based diversity.

“Epistemic diversity, while sometimes overlapping with previously mentioned notions of group diversity, is conceptually distinct.”

Probably not a substantial claim: may not pass S4 (based).

In finite domains, the answer space may be approximated through a curated reference set, allowing enumeration-based coverage metrics.

“In finite domains, the answer space may be approximated through a curated reference set, allowing enumeration-based coverage metrics.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Zollman (2010) argues that premature convergence is itself an epistemic risk: communities that converge too quickly can settle early on an inferior theory, whereas maintaining a transient diversity of positions improves long-run outcomes.

“Zollman (2010) argues that premature convergence is itself an epistemic risk: communities that converge too quickly can settle early on an inferior theory, whereas maintaining a transient diversity of positions improves long-run outcomes.”

• cites Zollman (2010)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Accuracy, fairness, and alignment remain essential, but they do not fully capture the behavior of systems increasingly used for explanation, tutoring, writing, and inquiry.

“Accuracy, fairness, and alignment remain essential, but they do not fully capture the behavior of systems increasingly used for explanation, tutoring, writing, and inquiry.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Task-specific proxies such as Wikipedia reference lists and proof categories are themselves incomplete and may reflect existing biases.

“We therefore rely on task-specific proxies such as Wikipedia reference lists and proof categories, which are themselves incomplete and may reflect existing biases.”

Probably not a substantial claim: may not pass S4 (based).

Analogous comparisons can be made between accessible, multi-turn, multi-output, and latent protocols to assess how much additional diversity becomes available when diversity-seeking intent is made explicit.

“Analogous comparisons can be made between accessible, multi-turn, multi-output, and latent protocols to assess how much additional diversity becomes available when diversityseeking intent is made explicit.”

May combine two claims.

Epistemic diversity is sensitive to prompting, decoding, personalization, and interaction design, making it a property of both the model and the interface through which users encounter it.

“Third, epistemic diversity is sensitive to prompting, decoding, personalization, and interaction design, making it a property of both the model and the interface through which users encounter it.”

May combine two claims.

Epistemic diversity is both conceptually important and practically measurable.

“Despite these limitations, our results show that epistemic diversity is both conceptually important and practically measurable.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Work in political science further argues that preserving multiple perspectives matters even when current evidence favors one solution, because the evidence itself may be incomplete and subject to change.

“Work in political science further argues that preserving multiple perspectives matters even when current evidence favors one solution, because the evidence itself may be incomplete and subject to change (M üller, 2023).”

• cites M üller, 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S4 (based).

High invalidity or hallucination rates point instead to failures of validity, extraction, or grounding.

“High invalidity or hallucination rates point instead to failures of validity, extraction, or grounding.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Recent work has begun exploring multiple-answer evaluation settings.

“Recent work has begun exploring multiple-answer evaluation settings (Xu et al., 2025b).”

• cites Xu et al., 2025bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Wright et al. (2025) found that newer models show improvements in diversity of factual claims but lower diversity than web search.

“Wright et al. (2025) measure diversity via variation in factual claims, finding improvements in newer models but lower diversity than web search.”

• cites Wright et al. (2025)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

This procedure distinguishes epistemically meaningful diversity from surface variation and invalid outputs, and enables comparison across interaction protocols, separating the diversity users see by default from the broader diversity a model can reveal under stronger probing.

“This procedure distinguishes epistemically meaningful diversity from surface variation and invalid outputs.It also enables comparison across interaction protocols, separating the diversity users see by default from the broader diversity a model can reveal under stronger probing.”

Classified as probably not a substantial claim.

For correctly specified, finite tasks, epistemic diversity can be measured through coverage over an enumerable reference set.

“For the fourth quadrant (correctly specified, finite tasks), epistemic diversity can be measured through coverage over an enumerable reference set.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Answer spaces are often ambiguous or only partially observable.

“First, answer spaces are often ambiguous or only partially observable.”

Classified as probably not a substantial claim. May combine two claims.

Because this form of probing is not meant to simulate typical user behavior, we additionally consider more naturalistic diversity-seeking protocols: a multi-turn protocol, in which the user requests alternatives in follow-up turns, and a multi-output protocol, in which the user asks for several answers in a single turn.

“Because this form of probing is not meant to simulate typical user behavior, we additionally consider more naturalistic diversity-seeking protocols: a multi-turn protocol π mt , in which the user requests alternatives in follow-up turns, and a multi-output protocol π mo , in which the user asks for several answers in a single turn.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

Choosing the threshold τ is difficult in practice and cannot be universally prescribed, much like similar thresholds in fairness literature.

“Yet choosing this value is difficult in practice and cannot be universally prescribed, much like similar thresholds in fairness literature (Barocas et al., 2023).”

• cites Barocas et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Xu et al. (2025a) observed repeated patterns in story generation.

“Xu et al. (2025a) observe repeated patterns in story generation.”

• cites Xu et al. (2025a)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S4 (based).

The latent protocol provides a more controlled probing setup that approximates an upper bound on recoverable diversity, by iteratively asking for new valid answers while excluding those already produced.

“The latent protocol π lat provides a more controlled probing setup that approximates an upper bound on recoverable diversity, by iteratively asking for new valid answers while excluding those already produced.”

Probably not a substantial claim: may not pass S2 (checkable).

Additional user context can legitimately narrow the valid answer space.

“Additional user context can legitimately narrow the valid answer space.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Feminist epistemologists emphasize the role of standpoints, arguing that knowledge is always situated and that inclusive inquiry requires acknowledging diverse positions and modes of reasoning.

“Feminist epistemologists emphasize the role of standpoints, arguing that knowledge is always situated and that inclusive inquiry requires acknowledging diverse positions and modes of reasoning (Harding, 1991;Collins, 2002).”

• cites Harding, 1991Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Collins, 2002Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

For underspecified, infinite tasks, evaluation is harder because both user intent and answer validity are less fixed; diversity likely needs to be measured over answer dimensions, clusters, themes, or reasoning approaches, using human or LLM judgments.

“For the third quadrant (underspecified, infinite tasks), evaluation is harder because both user intent and answer validity are less fixed.In these cases, diversity likely needs to be measured over answer dimensions, clusters, themes, or reasoning approaches, using human or LLM judgments.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Collective intelligence research shows that groups perform better when they incorporate cognitively diverse perspectives, even when individuals are less accurate on average.

“Similarly, in the social sciences, collective intelligence research shows that groups perform better when they incorporate cognitively diverse perspectives, even when individuals are less accurate on average (Page, 2007).”

• cites Page, 2007Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Diversity is not only about fair representation of people, but also about preserving multiple valid ways of knowing, interpreting, and reasoning.

“Taken together, these traditions suggest that diversity is not only about fair representation of people, but also about preserving multiple valid ways of knowing, interpreting, and reasoning.”

May combine two claims.

Most LLM benchmarks implicitly assume a single correct or preferred answer, treating diversity as noise or variability.

“Most LLM benchmarks implicitly assume a single correct or preferred answer (Hendrycks et al., 2021), treating diversity as noise or variability.”

• cites Hendrycks et al., 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S4 (based).

The accessible protocol approximates the range of valid answers surfaced under ordinary prompting, limited resampling, or repeated exposure with the same prompt across users.

“The accessible protocol π acc approximates the range of valid answers surfaced under ordinary prompting, limited resampling, or repeated exposure with the same prompt across users.”

Probably not a substantial claim: may not pass S2 (checkable).

Measuring validity and answer equivalence is difficult, especially in open-ended or subjective domains.

“Second, measuring validity and answer equivalence is difficult, especially in open-ended or subjective domains.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Low latent diversity may indicate model knowledge limitations, weak recoverability, or insufficient task coverage.

“Low latent diversity may indicate model knowledge limitations, weak recoverability, or insufficient task coverage.”

Classified as probably not a substantial claim.

The Professions task highlights a finite but rich answer space: well-known figures whose existence and relevance are verifiable (e.g., via Wikipedia).

“This task highlights a finite but rich answer space: well-known figures whose existence and relevance are verifiable (e.g., via Wikipedia).”

Probably not a substantial claim: may not pass S2 (checkable).

Epistemic pluralism holds that scientific and social progress depends on the coexistence of multiple, sometimes competing, perspectives.

“Epistemic pluralism holds that scientific and social progress depends on the coexistence of multiple, sometimes competing, perspectives (Longino, 2002;Feyerabend, 2002;Kellert et al., 2006).”

• cites Longino, 2002Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Feyerabend, 2002Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Kellert et al., 2006Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Goethals & Rhue (2025) documented the superstar effect, where models repeatedly surface a small set of prominent individuals.

“Goethals & Rhue (2025) document the superstar effect, where models repeatedly surface a small set of prominent individuals.”

• cites Goethals & Rhue (2025)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

The prompt formats are not intended as independent task settings; they vary the generation format while holding the underlying question fixed, so that we can test whether concentration persists across common generation formats rather than being an artifact of one prompt phrasing.

“These formats are not intended as independent task settings.They vary the generation format while holding the underlying question fixed (i.e., which individual the model treats as salient for a given profession), so that we can test whether concentration persists across common generation formats rather than being an artifact of one prompt phrasing.”

Classified as probably not a substantial claim.

The Professions dataset corresponds to the underspecified, finite setting.

“Professions.This dataset corresponds to the underspecified, finite setting.”

Classified as probably not a substantial claim.

Our automated annotation pipeline should be understood as an approximation rather than a definitive oracle, even when supported by human validation.

“Our automated annotation pipeline should therefore be understood as an approximation rather than a definitive oracle, even when supported by human validation (see Appendix E.6).”

May combine two claims.

This is appropriate for factual tasks, but fails in settings where multiple answers are equally valid.

“This is appropriate for factual tasks, but fails in settings where multiple answers are equally valid.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Wikipedia is not a perfect or unbiased baseline, but it covers a wide array of professionals who are generally accepted as being well-known.

“We note that Wikipedia is not a perfect or unbiased baseline, but it covers a wide array of professionals who are generally accepted as being well-known.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

We treat epistemic diversity as a distinct capability, capturing whether models express substantively different valid knowledge.

“In contrast, we treat epistemic diversity as a distinct capability, capturing whether models express substantively different valid knowledge.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Our experiments operationalized only two regions of the proposed framework and should be read as controlled case studies rather than an exhaustive benchmark.

“Finally, our experiments operationalized only two regions of the proposed framework and should be read as controlled case studies rather than an exhaustive benchmark.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Rather than fix a universal τ, we treat it as a flexible threshold: a way to reason about model coverage relative to plausible answer spaces.

“Rather than fix a universal τ, we treat it as a flexible threshold: A way to reason about model coverage relative to plausible answer spaces.”

May combine two claims.

Models tend to converge to narrow regions of the valid answer space, under-representing both entities and reasoning strategies.

“These findings show that models tend to converge to narrow regions of the valid answer space, under-representing both entities and reasoning strategies.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Wikipedia contains lists of well-known practitioners from a profession.

“Wikipedia contains lists of well-known practitioners from a profession, e.g., https://en.wikipedia. org/wiki/List of physicists.”

• cites https://en.wikipedia. org/wiki/List of physicistsNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Exclusion-based latent probing drives different models along the same long tail of the reference set, so their outputs overlap heavily, whereas the other protocols produce more model-specific sets.

“Exclusion-based latent probing drives different models along the same long tail of the reference set, so their outputs overlap heavily, whereas the other protocols produce more model-specific sets.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Demanding high epistemic diversity risks generating irrelevant or redundant outputs, while setting τ too low can reinforce epistemic monocultures.

“Demanding high epistemic diversity risks generating irrelevant or redundant outputs, while setting τ too low can reinforce epistemic monocultures.”

Classified as probably not a substantial claim.

WildChat is a corpus of 1M real-world user interactions with ChatGPT.

“WildChat is a corpus of 1M real-world user interactions with ChatGPT.”

• cites Zhao et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

Protocol comparisons are paired and analyzed using the Wilcoxon signed-rank test with the model as the unit of analysis (n = 10), which makes no distributional assumption and avoids treating the 360 model-profession-format cells as independent.

“Because every model is evaluated under every protocol, protocol comparisons are paired.We therefore use the Wilcoxon signed-rank test with the model as the unit of analysis (n = 10), which makes no distributional assumption and avoids treating the 360 model-profession-format cells as independent.”

Probably not a substantial claim: may not pass S2 (checkable).

These results reinforce the need for epistemic diversity-aware evaluation.

“These results reinforce the need for epistemic diversity-aware evaluation.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The bootstrap confidence intervals reflect variation across prompts only and do not capture uncertainty from the choice of reference list or from the annotation judge, which are treated as fixed.

“These intervals reflect variation across prompts only.They do not capture uncertainty from the choice of reference list or from the annotation judge, which we treat as fixed.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Wikipedia lists 12 distinct proofs for the Euclidean theorem (as of January 2026), while the underlying literature documents at least 200.

“E.g., Wikipedia lists 12 distinct proofs for the Euclidean theorem (as of January 2026), while the underlying literature documents at least 200 (Meštrović, 2023).”

• cites Meštrović, 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

To assess the reliability of the LLM-based annotation pipeline, two authors independently annotated 100 randomly selected outputs: 50 from the professions task and 50 from the proofs task, with 5 samples from each of the 10 tested models per task.

“To assess the reliability of our LLM-based annotation pipeline, two authors independently annotated 100 randomly selected outputs: 50 from the professions task and 50 from the proofs task, with 5 samples from each of the 10 tested models per task.”