The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index; 3 of the 52 sources its claims cite were checked for support.
When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content
Open accessAbstract
Generative artificial intelligence (GenAI) blurs the boundaries between expert and non-expert sources, as it increasingly distributes and creates scientific content. This study examines how individuals adapt evaluation strategies, including content and source evaluation, and corroboration, when using GenAI versus a search engine. Based on performance tasks in which participants evaluated science-related socio-scientific dilemmas and follow-up interviews with 30 adult participants from diverse educational backgrounds, findings reveal that users employed these strategies on both platforms but adapted them in distinct ways. We identified two evaluation strategies that emerged as analytical constructs from the qualitative data. First, to corroborate output, participants frequently used a strategy we titled ‘representation evaluation,’ assessing whether GenAI accurately summarized its sources rather than verifying source agreement independently. Second, participants also applied ‘meta source evaluation,’ relying on their familiarity with sources provided by GenAI instead of directly evaluating the sources themselves. Although all participants engaged in dialogue with the chat, they did not leverage the bot’s dialogue capabilities to assess credibility, and many relied on a “machine heuristic”, assuming GenAI’s inherent correctness, reflecting a well-documented over-trust in automated systems. This research underscores the importance of developing and assessing critical evaluation skills for navigating AI-generated scientific information. Specifically, it extends existing models of online information evaluation to contexts mediated by artificial intelligence.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This qualitative study examines how 30 adults evaluated science-related socio-scientific dilemmas using Bing-Chat versus Google Search. It identifies adapted evaluation strategies—representation evaluation and meta-source evaluation—alongside persistent biases, especially machine heuristic in the GenAI condition.
Key findings
- Content evaluation was similar across platforms: 25 of 30 participants assessed plausibility/prior knowledge on Google, and 27 of 30 did so on Bing-Chat; none proactively sought scientific consensus during the tasks.
- For corroboration on Bing-Chat, half of participants clicked the sources provided to check whether the AI accurately summarized them, a strategy the authors call “representation evaluation,” rather than independently verifying source agreement.
- Source evaluation shifted to “meta-source evaluation”: 12 of 30 judged Bing-Chat by the appropriateness or familiarity of the sources it cited without directly evaluating those sources; all 30 engaged in dialogue with the chat but did not use dialogue to assess credibility.
- The same 16 participants showed confirmation bias on both platforms, while a dozen displayed the machine heuristic only when using Bing-Chat.
Methods
- Qualitative design combining two Zoom performance tasks, observation, and hour-long semi-structured interviews with 30 adult participants from diverse educational, scientific, and algorithmic backgrounds; data were collected 19/06/2023–20/08/2023.
- Participants evaluated two socio-scientific issue vignettes—a shared detox-diet task and a self-selected topic from ten options—using Bing-Chat and Google Search in counterbalanced order, with recommendations requested three times per task.
- Reflexive thematic analysis in ATLAS.ti used directed and conventional coding, triangulating observation and interview data, with peer debriefing and consensus meetings.
Limitations
- The paper states that interpretive coding reflects researchers’ analytic lenses; the lab setting may increase effort and expectation bias; much data rely on self-reports; the small sample does not support generalization; most participants had little prior GenAI experience; and the single-country, narrow-age sample limits transferability.
- The paper also states that Bing-Chat’s specific affordances limit generalizability to other GenAI systems and that it cannot draw conclusions about correlations between scientific/algorithmic knowledge and evaluation ability.
- Evident from the text: the 2023 data and platform-specific findings may not reflect current GenAI interfaces or citation behaviors, and the study reports no quantitative comparative tests of platform effects.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 of 52 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured
Every source the paper cites for this claim was checked and supports it.
In the paper
“The contrast between Google and Bing-Chat can be better understood through the lens of interface affordances, which shape not only how users access information but also how they interpret and trust it [29]”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.2 Platform interface shapes user evaluation processes
What the sources say
- Supportedcites [29] N Hirvonen, 2023, Artificial intelligence in the information ecosystem: affordances for everyday information seeking, J Assoc Inf Sci Technol source
Section 2.1 describes affordances as ways information systems enable and constrain practices. Section 3.2 contrasts traditional search results with generative AI answers and explains how hidden sources can hamper credibility evaluation; Section 4.2 also links information-system conditions to trust and credibility. Applying that framework to Google and Bing-Chat is the citing authors’ framing, not a comparison attributed to this source. (LLM, conf=0.87)
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, to corroborate output, participants frequently used a strategy we titled ‘representation evaluation,’ assessing whether GenAI accurately summarized its sources rather than verifying source agreement independently.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Evaluation of information based on the degree of agreement with expert knowledge was not part of the search process of any participant.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, participants also applied ‘meta source evaluation,’ relying on their familiarity with sources provided by GenAI instead of directly evaluating the sources themselves.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Evaluating the site type was particularly evident during observations, where 23 participants accessed official sources such as healthcare providers, the Ministry of Health, or academic articles, even when those were not the first search results.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Strength:plain· not yet calibratedn = 23Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When cognitive biases were at play, however, the differences between the platforms blurred: participants who relied on cognitive biases in Google did so also on Bing-Chat.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found expressions of all the critical evaluation strategies when using Bing-Chat.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“During the observations, we saw that all participants consulted at least three different sources when searching with Google.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Evaluating the content’s author was discussed more in the interviews than observed.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Strength:plain· not yet calibratedn = 4Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When using Google, participants employed many of the critical evaluation strategies in ways that mirrored recommended practices found in the literature to examine specific websites appearing in search results.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Probably not a substantial claim. 56
- It may state two separate claims as one. 56
- The quote only partly states this claim. 1
Every source the paper cites for this claim was checked and supports it.
In the paper
“AI systems construct dynamic environments of affordances that both enable and constrain user cognition [29].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.2 Platform interface shapes user evaluation processes
What the sources say
- Supportedcites [29] N Hirvonen, 2023, Artificial intelligence in the information ecosystem: affordances for everyday information seeking, J Assoc Inf Sci Technol source
The Discussion (section 4, page unavailable) describes an environment of affordances created as AI tools enter existing practices, with systems that adapt to users and both expand and restrict information seeking. Section 4.2 (page unavailable) says these tools shape actions and thinking, supporting the claim’s reference to user cognition. (LLM, conf=0.88)
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Dialogue enhances the quality of AI-provided information and support by personalizing interactions based on dialogue history and adaptive responses [64].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.3 The missing heuristic: An ongoing dialogue with GenAI about its answers and sources
What the sources say
- Partly supportedcites [64] M Milne-Ives, 2020, The effectiveness of artificial intelligence conversational agents in health care: systematic review, J Med Internet Res sourceExperimental — accuracy not yet measured
The Introduction (Background; page unavailable) says conversational agents can respond to user input and that personalizable features may improve usability and satisfaction. It does not establish that dialogue improves the quality of information or support through dialogue history and adaptive responses; it also frames the benefit of personalization as a potential, not a demonstrated effect. (LLM, conf=0.83)
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants applied similar evaluation strategies when using both Google and Bing-Chat, but the implementation of these strategies differed between the platforms, with GenAI users demonstrating different tendencies than those described by the literature.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Google's centrality as a global information retrieval tool makes participants' performance within Google a valid baseline for comparison.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Only content evaluation, however, was conducted similarly to Google, while corroboration and source evaluation strategies were implemented differently.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although all participants engaged in dialogue with the chat, they did not leverage the bot’s dialogue capabilities to assess credibility”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The findings show that while participants employed these evaluation strategies in both Google and Bing-Chat, their application of these methods varied considerably between the two technologies (Fig 2).”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Google's centrality as a global information retrieval tool makes participants' performance within Google a valid baseline for comparison.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A large majority of participants (25 of 30) relied on prior knowledge and assessed the plausibility of the information.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the interviews, it was seen that site evaluation was based on familiarity with the specific site (27 participants), and awareness of commercial interests (26 participants).”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Strength:plain· not yet calibratedn = 27Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All participants mentioned the strategy in their interviews and explained their preference for cross-referencing.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This was observed during Google searches in two ways: (1) evaluating the site or type of site, e.g., media source, or the Ministry of Health website; (2) evaluating the content’s author, e.g., author’s professional credentials.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Based on performance tasks in which participants evaluated science-related socio-scientific dilemmas and follow-up interviews with 30 adult participants from diverse educational backgrounds, findings reveal that users employed these strategies on both platforms but adapted them in distinct ways.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > Abstract
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that none of our participants used it, although it was presented to them during the familiarizing session before the main task (Fig 1).”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.3 The missing heuristic: An ongoing dialogue with GenAI about its answers and sources
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, Bing-Chat users first encounter the synthesized content, followed by a limited list of cited sources”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.2 Platform interface shapes user evaluation processes
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: ChatGPT typically provides no citations at all.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although users were often found to focus only on the first few links on Google and demonstrate high confidence in its ranking system”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.2 Platform interface shapes user evaluation processes
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Evaluation of the content was similar across both platforms, but corroboration and source evaluation took on a different form when applied to Bing-Chat.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.1 Adapting existing evaluation strategies for assessing AI-generated information
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Google's centrality as a global information retrieval tool makes participants' performance within Google a valid baseline for comparison.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Consequently, unlike when using Google, most participants initially evaluated the information itself rather than its source.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.2 Platform interface shapes user evaluation processes
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When participants sought information within a chat interface, they often relied on the chat’s responses without deeply investigating the sources and also without interrogating the bot about its sources, their credibility, and how it chose them (Fig 3).”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.3 The missing heuristic: An ongoing dialogue with GenAI about its answers and sources
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All participants used critical evaluation strategies, though some also relied on cognitive biases.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.4 Bias awareness: A missing element in science education
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When using Google, participants evaluated the expertise and integrity of specific websites and their authors; in Bing-Chat, however, participants engaged in what we term ‘meta-source evaluation.’”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.1 Adapting existing evaluation strategies for assessing AI-generated information
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Google's centrality as a global information retrieval tool makes participants' performance within Google a valid baseline for comparison.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All participants employed critical evaluation strategies, but some also relied on cognitive biases when assessing the information.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.2 Reliance on cognitive biases in the two technologies
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While participants trusted Google to provide quality sources in the high ranked results it provides, they did scroll down and review the other ones, deciding which to view.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.2 Reliance on cognitive biases in the two technologies
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“After selecting a source, evaluating the information becomes more nuanced and differs across users.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.2 Platform interface shapes user evaluation processes
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although all participants engaged in dialogue with the chat, each asking at least two questions related to the topic under investigation, and many asking considerably more”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.3 The missing heuristic: An ongoing dialogue with GenAI about its answers and sources
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“They evaluated the GenAI as a source based on the credibility of the sources it listed, but without evaluating these sources themselves.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.1 Adapting existing evaluation strategies for assessing AI-generated information
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The machine heuristic was identified solely during evaluating information provided by Bing-Chat and was not observed when assessing information on Google.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.2 Reliance on cognitive biases in the two technologies
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unlike their behaviour on Google, where they often questioned the reliability of sources or compared information across multiple results, in the chat environment, they tended to accept the responses as given.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.3 The missing heuristic: An ongoing dialogue with GenAI about its answers and sources
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the respondents in this study also relied on two main biases: confirmation bias, seen in both Google and Bing-Chat, and the machine heuristic, observed only in Bing-Chat.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.4 Bias awareness: A missing element in science education
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“It acknowledges the epistemic dependence of non-experts, which refers to their reliance on trusted intermediaries because they cannot directly verify the accuracy of scientific claims [6].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 1. Introduction
What the sources say
- Not checked yetcites [6] J Osborne, 2024, Science literacy in the twenty-first century: informed trust and the competent outsider, Int J Sci Educ source
Checked before — its earlier verdict is held back until a new check runs.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- Not checked yet
In the paper
“To address ordering effects, which occur ‘when the order of presented information alters user perceptions and decisions’ [53], those participants who searched first on Bing-Chat in the first part now searched first on Google (and vice versa).”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.3 Data collection and research tools
What the sources say
- Not checked yetcites [53] S Passi, 2022
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“By contrast, System 2 thinking encourages deliberate, reflective analysis, which includes slowing down to question, verify, and compare information [3].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 1. Introduction
What the sources say
- Not checked — no full text of the sourcecites [3] I Tabak, 2020, Post-truth GPS: Detour at truth, take the long route to useful knowledge, Educ Psychol source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“We note that the prominent use of proactive corroboration might have been influenced by the research setting, possibly due to biases related to social interaction during the study [50].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
What the sources say
- Not checked — no full text of the sourcecites [50] M Allen, 2017 source
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Not only that ‘flattening of expertise’ blurs the distinction between expert and non-expert voices, but it is also accompanied by an increasing user’ dependence on opaque systems whose reasoning they cannot inspect.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 1. Introduction
What the sources say
- Not matchedcites [7] Author. 26th Annual Conference of the ISCA; 2024.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“reflecting a well-documented over-trust in automated systems.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > Abstract
Strength:boosted· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“The literature identifies several evaluation strategies that require deeper cognitive engagement, including content evaluation, source evaluation, and corroboration with other credible sources [4,5].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 1. Introduction
What the sources say
- Not checked — no full text of the sourcecites [4] S Barzilai, 2020, Making sources visible: promoting multiple document literacy with digital epistemic scaffolds, Comput Educ source
- Not checked — no full text of the sourcecites [5] R Bromme, 2014, The public’s bounded understanding of science, Educ Psychol source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“At that time, ChatGPT3.5 did not function optimally in [local language], did not provide references, and was limited in providing current information since its corpus was up to date only through 2021 [45].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.1 GenAI platform choice
What the sources say
- Not checked yetcites [45] J Rudolph, 2023, War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education, J Appl Learn Teach
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The small quota groups (n = 5) were not used for comparative analyses but rather functioned as a sampling frame to ensure heterogeneity in participants’ educational, scientific, and algorithmic backgrounds.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.2 Research population and research sample
Strength:plain· not yet calibratedn = 5Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Recent methodological reviews indicate that meaning saturation in semi-structured interviews is typically reached after approximately 24 interviews, with theoretical saturation commonly achieved within 20–30 interviews across grounded and thematic designs [51].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.2 Research population and research sample
What the sources say
- Not checked — no full text of the sourcecites [51] A Wutich, 2024, Sample sizes for 10 types of qualitative data analysis: an integrative review, empirical guidance, and next steps, Int J Qual Methods source
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“This format balances a guiding framework with flexibility, enabling both broad understanding and in-depth insights [50] (see S6 File).”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.3 Data collection and research tools
What the sources say
- Not checked — no full text of the sourcecites [50] M Allen, 2017 source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The use of Bing-Chat increases ecological validity by reflecting the available technological conditions at the time of data collection, but also poses a limitation to generalizability.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.1 GenAI platform choice
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The research team held multiple consensus meetings at key stages of the analysis, once during coding and again during the synthesis of findings. When interpretive differences arose, we examined their sources in depth and resolved them through discussion until a complete consensus was achieved.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.5 Trustworthiness
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“In this approach, themes are actively constructed through iterative engagement with the data, and analytic rigor derives from reflexivity, theoretical coherence, and depth rather than from coder agreement indices.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.4 Data analysis
What the sources say
- Not checked — no full text of the sourcecites [54] V Braun, 2022, Conceptual and design thinking for thematic analysis, Qual Psychol source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“These include the opacity of its algorithms, potential contamination or bias in its data sources [8], and being prone to ‘hallucinations,’ the generation of syntactically and contextually correct, reasoned, and confident answers that are nonetheless incorrect [9].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 1. Introduction
What the sources say
- Not checked yetcites [8] A Bandi, 2023, The power of generative AI: a review of requirements, models, input–output formats, evaluation metrics, and challenges, Future Internet source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“In online settings that favour speed and minimal effort, users often rely on System 1 thinking, making quick credibility judgments that are prone to cognitive biases such as confirmation bias, the tendency to favour information that supports existing beliefs while ignoring contradictions [2,12].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 1. Introduction
What the sources say
- Not checked — no full text of the sourcecites [2] MA Britt, 2019, A reasoned approach to dealing with fake news, Policy Insights Behav Brain Sci source
- Not checked — no full text of the sourcecites [12] S Valenzuela, 2022, Social media and belief in misinformation in Mexico: a case of maximal panic, minimal effects?, Int J Press/Politics source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Peer debriefing was an integral part of the research process [57].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.5 Trustworthiness
What the sources say
- Not checked yetcites [57] VN Anney, 2014, Ensuring the quality of the findings of qualitative research: looking at trustworthiness criteria, JETERAPS
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“Google Search (hereafter Google) was the chosen search engine, as it is the most widely used search engine globally, with a market share (and dominance) of approximately 82% [46] and 98% in [country] [47].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.1 GenAI platform choice
What the sources say
- Not matchedcites [46] Statista
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [47] Statcounter GS
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
In the paper
“Through iterative analytic meetings, the authors refined the coding scheme by merging, renaming, or discarding codes based on conceptual clarity, distinctiveness, and alignment with the study’s interpretive aims, consistent with Braun and Clarke’s [54] principle of design coherence.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.4 Data analysis
What the sources say
- Not checked — no full text of the sourcecites [54] V Braun, 2022, Conceptual and design thinking for thematic analysis, Qual Psychol source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“but also poses a limitation to generalizability. GenAI systems differ in citation behavior, transparency, and interface design, which may influence users’ evaluation processes.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.1 GenAI platform choice
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: GenAI systems differ in citation behavior, transparency, and interface design, which may influence users’ evaluation processes.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All thirty participants engaged in conversation with the chat, each asking at least two questions related to the topic under investigation, and many asking considerably more questions.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Specifically, it extends existing models of online information evaluation to contexts mediated by artificial intelligence.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
In the paper
“To address anchoring effect, which is ‘relying too much on the first piece of provided information when making decisions’ [53], we asked participants for their opinion before asking them to search for information.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.3 Data collection and research tools
What the sources say
- Not checked yetcites [53] S Passi, 2022
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“All thirty participants were included in every stage of analysis; references to smaller numbers throughout this section reflect how many participants demonstrated or mentioned a particular behaviour or theme rather than any exclusion from the dataset.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“These include the opacity of its algorithms, potential contamination or bias in its data sources [8], and being prone to ‘hallucinations,’ the generation of syntactically and contextually correct, reasoned, and confident answers that are nonetheless incorrect [9].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 1. Introduction
What the sources say
- Not checked yetcites [8] A Bandi, 2023, The power of generative AI: a review of requirements, models, input–output formats, evaluation metrics, and challenges, Future Internet source
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“Triangulation, by combining observations and interviews, helped to minimize bias and assess the comprehensiveness of participants’ responses [56] while increasing analytical validity.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.5 Trustworthiness
What the sources say
- Not checked — no full text of the sourcecites [56] AK Shenton, 2004, Strategies for ensuring trustworthiness in qualitative research projects, Educ Inf source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“In addition, throughout data collection, analysis, and interpretation, we consulted impartial colleagues who are experts in science communication and not co-authors of this paper. These experts reviewed the initial findings and preliminary analysis, providing critical feedback and guidance to help the researcher maintain integrity and foster a more reflective analysis [57]. Their input challenged”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.5 Trustworthiness
What the sources say
- Not checked yetcites [57] VN Anney, 2014, Ensuring the quality of the findings of qualitative research: looking at trustworthiness criteria, JETERAPS
- Not checked — no full text of the sourcecites [58] JW Creswell, 2000, Determining validity in qualitative inquiry, Theory Into Pract source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In sum, the use of critical evaluation strategies—content assessment, corroboration, and source verification—was prevalent and effectively integrated when searching on Google.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.1 Critical evaluation strategies applied in the two technologies
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“According to dual process theory, people evaluate information using two systems: the intuitive and fast “System 1,” and the slower, more analytical “System 2” [1].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 1. Introduction
What the sources say
- Not checked yetcites [1] D Kahneman, 2011
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Due to the unexplored nature of the research field, we aimed to minimize the influence of researchers’ preconceptions and expectations by employing a semi-structured interview format.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.3 Data collection and research tools
Tests✗ Asserted: fails✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. The authors may not put it forward as true.
In the paper
“While research has investigated how people evaluate GenAI based on its fairness, responsibility, and transparency [10], studies examining the evaluation strategies used by people when engaging with GenAI are rare.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 1. Introduction
What the sources say
- Not checked — no full text of the sourcecites [10] D Shin, 2024, Impact of misinformation from generative AI on user information processing: how people understand misinformation from generative AI, New Media Soc source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“At the time of data collection, it communicated well in the local language, had access to the internet enabling it to offer current and updated information and cited its sources [45].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 3. Materials and methods > 3.1 GenAI platform choice
What the sources say
- Not checked yetcites [45] J Rudolph, 2023, War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education, J Appl Learn Teach
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The move from a retrieval-based architecture (Google) to a generative, synthesized interface (Bing-Chat) reshaped not only how information was accessed but also how credibility was constructed.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.1 Adapting existing evaluation strategies for assessing AI-generated information
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Public understanding of GenAI is slowly emerging, as people begin to use these technologies [49].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.4 Bias awareness: A missing element in science education
What the sources say
- Not matchedcites [49] Author. Public Communication of Science and Technology (PCST) China Symposium 2024; 2024.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
In the paper
“This presentation structure fosters the machine heuristic, the tendency to over-rely on outputs perceived as competent or authoritative [14], thereby encouraging users to delegate elements of critical evaluation to the AI itself.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.2 Platform interface shapes user evaluation processes
What the sources say
- Not checked — no full text of the sourcecites [14] Sundar SS, Kim J. Machine heuristic: when we trust computers more than humans with our personal information. In: Brewster SA, Fitzpatrick G, Cox AL, Kostakos V, editors. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems; 2019 May 4–9; Glasgow, UK; ACM; 2019. p. 1–9. source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although participants described relying on evaluation strategies across both technological environments, these strategies did not transfer unchanged to the GenAI context.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.1 Adapting existing evaluation strategies for assessing AI-generated information
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These have not been previously described in the literature and are revealed here due to the unique intersection of cognitive and sociotechnical inquiry.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.1 Adapting existing evaluation strategies for assessing AI-generated information
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Most participants had little prior experience with GenAI, so their strategies may reflect initial experimentation rather than stable evaluation practices.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.5 Limitations and way forward
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“First, given the interpretative nature of qualitative coding, the findings inevitably reflect the researchers’ analytic lenses and theoretical commitments, even though systematic procedures were employed to enhance credibility.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.5 Limitations and way forward
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This bias might arise from the characteristics of the chat interface, as the chat itself generates the content.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.2 Reliance on cognitive biases in the two technologies
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bing-Chat’s generative architecture cultivated a mono-source illusion, positioning the AI itself as an epistemic gatekeeper.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.2 Platform interface shapes user evaluation processes
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, the findings indicated that while participants employed established evaluation strategies across both platforms, their application was reshaped by the affordances of GenAI.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.2 Reliance on cognitive biases in the two technologies
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Together, these patterns demonstrated both continuity and transformation in users’ evaluative practices, highlighting how GenAI environments reconfigure traditional approaches to assessing information credibility.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 4. Results > 4.2 Reliance on cognitive biases in the two technologies
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Consequently, Bing-Chat transformed not only users’ search behaviours but also their patterns of trust.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.2 Platform interface shapes user evaluation processes
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“This finding aligns with Kresin et al. [59], who found that when evaluating scientific information on social media, participants cited familiarity with the source as a primary criterion for credibility.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.1 Adapting existing evaluation strategies for assessing AI-generated information
What the sources say
- Not checked yetcites Kresin et al. [59] S Kresin, 2024, Students’ credibility criteria for evaluating scientific information: the case of climate change on social media, Sci Educ source
Checked before — its earlier verdict is held back until a new check runs.
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Not checked yet
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This reliance on self-reporting is inevitable, as these processes involve internal thoughts that are not observable by an external observer.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.5 Limitations and way forward
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A similar risk could arise in GenAI, if it provides users with information based on such sources, since the selection of sources against which participants verified the data was not done independently.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.1 Adapting existing evaluation strategies for assessing AI-generated information
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Users might perceive Bing-Chat providing specific sources as an implicit endorsement of them, even though the technology does not necessarily verify their reliability.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.1 Adapting existing evaluation strategies for assessing AI-generated information
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
- Rests on an assumption: Bing-Chat does not necessarily verify the reliability of the sources it provides.
In the paper
“Furthermore, the performance task took place in a lab setting, likely increasing participant effort and introducing bias, as actions may align with perceived expectations [50].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.5 Limitations and way forward
What the sources say
- Not checked — no full text of the sourcecites [50] M Allen, 2017 source
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“These echoes recent work showing that people with minimal experience using GenAI were also less aware of its epistemic limitations [66].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.4 Bias awareness: A missing element in science education
What the sources say
- Not matchedcites [66] Author. The 75th Annual International Communication Association (ICA) Conference; 2025.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“Educating individuals to recognize these influences, alongside content and source evaluation, is critical for their epistemic vigilance [67,69].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.4 Bias awareness: A missing element in science education
What the sources say
- Not checked yetcites [67] T Bielik, 2024, Developing and evaluating the extended epistemic vigilance framework, J Res Sci Teach source
Checked before — its earlier verdict is held back until a new check runs.
- Not checked yetcites [69] Author, 2024, J Res Sci Teach
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- Not checked yet
In the paper
“Echo chambers that limit exposure to conflicting beliefs [2] may curate their own corpus of conspiratorial sources.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.1 Adapting existing evaluation strategies for assessing AI-generated information
What the sources say
- Not checked — no full text of the sourcecites [2] MA Britt, 2019, A reasoned approach to dealing with fake news, Policy Insights Behav Brain Sci source
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These novel adaptations reveal how practices of epistemic trust and verification are being reshaped in AI contexts, highlighting evolving processes of epistemic delegation and credibility transfer between users and generative systems.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 6. Conclusion
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, a significant portion of the data is based on the participants’ self-reports regarding their information evaluation processes.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.5 Limitations and way forward
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- Rests on an assumption: Participants’ self-reports accurately reflect their internal information evaluation processes.
In the paper
“GenAI has advanced the role of AI in dialogue-based applications [63].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.3 The missing heuristic: An ongoing dialogue with GenAI about its answers and sources
What the sources say
- Not checked — no full text of the sourcecites [63] J Zhang, 2024, Investigation of the effectiveness of applying ChatGPT in dialogic teaching using electroencephalography. source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We do not claim to generalize the frequency or prevalence of the strategies observed; rather, we highlight their existence.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.5 Limitations and way forward
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As epistemic responsibility is progressively delegated to algorithmic systems, established models of credibility assessment and verification in scientific communication are being reshaped, underscoring the need to reconceptualize these notions in AI-mediated knowledge environments.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 6. Conclusion
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
In the paper
“A key practice in information evaluation is lateral reading [21], which involves opening multiple tabs to verify a source’s credibility through independent web sources.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.3 The missing heuristic: An ongoing dialogue with GenAI about its answers and sources
What the sources say
- Not checked — no full text of the sourcecites [21] S Wineburg, 2019, Lateral reading and the nature of expertise: reading less and learning more when evaluating digital information, Teach Coll Rec source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Together, they offer nuance to the current theoretical understanding of how people negotiate authority, accountability, and reliability when the information source itself is algorithmic.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 6. Conclusion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, the study was conducted in a single country and focused on a relatively narrow age group, which may limit the transferability of the findings across cultural or demographic contexts.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.5 Limitations and way forward
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The novelty of the evaluation strategies presented here lies in their distinctive manifestations within an AI-mediated environment.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.1 Adapting existing evaluation strategies for assessing AI-generated information
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence.
In the paper
“These biases can lead individuals to fail to fully gather, internalize, or systematically process information, even when they have the ability to do so [1].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.4 Bias awareness: A missing element in science education
What the sources say
- Not checked yetcites [1] D Kahneman, 2011
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“As with the machine heuristic, awareness of this cognitive bias is essential, as it can help individuals to mitigate its effects [67].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.4 Bias awareness: A missing element in science education
What the sources say
- Not checked yetcites [67] T Bielik, 2024, Developing and evaluating the extended epistemic vigilance framework, J Res Sci Teach source
Checked before — its earlier verdict is held back until a new check runs.
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not checked yet
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition, given the nature of the study, we could not draw conclusions about the correlation between scientific and algorithmic knowledge and the ability to evaluate information across different platforms.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.5 Limitations and way forward
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, while generalizations are not warranted with small samples, such samples are appropriate for identifying and illustrating new phenomena, which is the primary contribution of our paper.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.5 Limitations and way forward
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The problem with this adaptation is that faulty answers may rely on summaries of unreliable sources, leading to unsupported claims and spreading misinformation.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.1 Adapting existing evaluation strategies for assessing AI-generated information
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“In addition, users are less likely to click on sources cited within the overview itself and are more likely to end their browsing session after viewing a page containing an AI overview.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.2 Platform interface shapes user evaluation processes
What the sources say
- Not matchedcites [61] A Chapekis, 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While Google’s search architecture preserves a distributed, source-based model of evaluation, prompting users to navigate among multiple actors and texts”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.2 Platform interface shapes user evaluation processes
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This controlled environment lacks the complexity of real-world contexts, where information is usually evaluated in response to personal needs.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.5 Limitations and way forward
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As such, other researchers might generate partially different categorizations from the same data.”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.5 Limitations and way forward
Strength:hedged· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“While knowledge of AI and its underlying mechanisms is critical to addressing the machine heuristic, awareness of this cognitive bias is key, as it empowers individuals to counteract it [67].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.4 Bias awareness: A missing element in science education
What the sources say
- Not checked yetcites [67] T Bielik, 2024, Developing and evaluating the extended epistemic vigilance framework, J Res Sci Teach source
Checked before — its earlier verdict is held back until a new check runs.
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not checked yet
In the paper
“Learning to recognize cognitive biases is crucial for scientific reasoning, yet it is often excluded from science curricula [68].”
§ When the source is a bot: How people adapt their evaluation strategies to assess AI-generated content > 5. Discussion > 5.4 Bias awareness: A missing element in science education
What the sources say
- Not checked — no full text of the sourcecites [68] D Allchin, 2022, Ten competencies for the science misinformation crisis, Sci Educ source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper relies on this without support.
Traditional models of credibility evaluation were developed for human-authored, source-transparent contexts, where cues such as identifiable authorship, institutional affiliation, or peer review help readers assess trustworthiness.
“Traditional models of credibility evaluation were developed for human-authored, source-transparent contexts, where cues such as identifiable authorship, institutional affiliation, or peer review help readers assess trustworthiness.”
The paper relies on this without support.
When information is produced or mediated by AI systems, the origins of data, the reasoning processes, and the identity of the 'source' are often opaque.
“When information is produced or mediated by AI systems, the origins of data, the reasoning processes, and the identity of the “source” are often opaque.”
The paper relies on this without support.
Studies examining the evaluation strategies used by people when engaging with GenAI are rare.
“While research has investigated how people evaluate GenAI based on its fairness, responsibility, and transparency [10], studies examining the evaluation strategies used by people when engaging with GenAI are rare.”
The paper relies on this without support.
Bing-Chat represents a generative-AI system that synthesizes and attributes information within a conversational interface, presenting its cited sources directly to the user, whereas Google Search functions as a retrieval platform that requires users to locate, navigate, and evaluate multiple external sources independently.
“Bing-Chat represents a generative-AI system that synthesizes and attributes information within a conversational interface, presenting its cited sources directly to the user, whereas Google Search functions as a retrieval platform that requires users to locate, navigate, and evaluate multiple external sources independently.”
The paper relies on this without support.
In both environments, users have access to sources, yet the modes of interaction and presentation differ substantially.
“In both environments, users have access to sources, yet the modes of interaction and presentation differ substantially.”
The paper relies on this without support.
GenAI systems differ in citation behavior, transparency, and interface design, which may influence users’ evaluation processes.
“GenAI systems differ in citation behavior, transparency, and interface design, which may influence users’ evaluation processes.”
The paper relies on this without support.
The source of information in GenAI is the GenAI itself, and its training data and content creation processes are often unknown to users.
“The emergence of GenAI, however, presents a challenge to these strategies, since the source of the information is the GenAI itself, and its training data and content creation processes are often unknown to the users.”
The paper relies on this without support.
Google's centrality as a global information retrieval tool makes participants' performance within Google a valid baseline for comparison.
“Given Google’s centrality as a global information retrieval tool, we use participants’ performance within this technology as a baseline.”
The paper relies on this without support.
Bing-Chat does not necessarily verify the reliability of the sources it provides.
“even though the technology does not necessarily verify their reliability.”
The paper relies on this without support.
The GenAI system's source is algorithmic and partly opaque.
“reflecting efforts to interpret information within a system where the “source” was algorithmic and partly opaque.”
The paper relies on this without support.
ChatGPT typically provides no citations at all.
“a feature not shared by all GenAI systems (e.g., ChatGPT, which typically provides no citations at all).”
The paper relies on this without support.
Google increasingly presents knowledge graphs, displaying content directly on the search page using AI.
“as Google increasingly presents knowledge graphs, displaying content directly on the search page using AI.”
The paper relies on this without support.
Participants’ self-reports accurately reflect their internal information evaluation processes.
“a significant portion of the data is based on the participants’ self-reports regarding their information evaluation processes.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Credibility transfer is described in opposite directions
MinorThe discussion first says participants used the perceived credibility of cited sources to judge Bing-Chat. It then describes credibility flowing from the AI interface to those sources, reversing the stated direction of the evaluation.
“They evaluated the GenAI as a source based on the credibility of the sources it listed, but without evaluating these sources themselves.”
“Meta-source evaluation represents processes of credibility transfer, in which the perceived trustworthiness of the AI interface is conferred onto the external sources it presents or cites.”
Citations
Cites
Three Approaches to Qualitative Content Analysis
Hsieh, H.-F., & Shannon, S. E. (2005) · Qualitative Health Research · doi:10.1177/1049732305276687 · Cited by 44,601
Strategies for ensuring trustworthiness in qualitative research projects
Shenton, A. K. (2004) · Education for Information · doi:10.3233/efi-2004-22201 · Cited by 9,748
Determining Validity in Qualitative Inquiry
Creswell, J. W., & Miller, D. L. (2000) · Theory Into Practice · doi:10.1207/s15430421tip3903_2 · Cited by 9,184
Conceptual and design thinking for thematic analysis.
Braun, V., & Clarke, V. (2022) · Qualitative Psychology · doi:10.1037/qup0000196 · Cited by 4,502 · cited 2×
“Using ATLAS.ti, we conducted a reflexive thematic analysis following Braun and Clarke’s [54] conceptualization of thematic analysis as an interpretive, researcher-generated process.”
Students’ voices on generative AI: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00411-8 · Cited by 2,301 · cited 1×
“Post-secondary students in Hong Kong struggled to assess the accuracy and validity of information generated by these tools [39].”
ChatGPT: five priorities for research
van Dis, E. A. M., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023) · Nature · doi:10.1038/d41586-023-00288-7 · Cited by 1,842 · cited 2×
“Furthermore, it enables the widespread circulation of misinformation disguised as credible scientific knowledge [30,31].”
SPERBER, D., CLÉMENT, F., HEINTZ, C., MASCARO, O., MERCIER, H., ORIGGI, G., & WILSON, D. (2010) · Mind & Language · doi:10.1111/j.1468-0017.2010.01394.x · Cited by 1,795
Pennycook, G., & Rand, D. G. (2021) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2021.02.007 · Cited by 1,360 · cited 1×
“When people pause and thoughtfully consider the information their search revealed, they can better identify misinformation, even if it conflicts with their preferred outcomes and beliefs [15,16].”
The SAGE Encyclopedia of Communication Research Methods
Allen, M. (2017) · doi:10.4135/9781483381411 · Cited by 1,103
Credibility and trust of information in online environments: The use of cognitive heuristics
Metzger, M. J., & Flanagin, A. J. (2013) · Journal of Pragmatics · doi:10.1016/j.pragma.2013.07.012 · Cited by 1,000
Flagged first, then by citations
Research works printed but not linked: 11
- D Kahneman, 2011, Thinking, fast and slow
- Author. 26th Annual Conference of the ISCA; 2024.
- Author, 2024, J Res Sci Teach
- W Holmes, 2023, Guidance for generative AI in education and research
- J Rudolph, 2023, War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education, J Appl Learn Teach, 6, 364
- Author. Public Communication of Science and Technology (PCST) China Symposium 2024; 2024.
- S Passi, 2022, Overreliance on AI: literature review
- VN Anney, 2014, Ensuring the quality of the findings of qualitative research: looking at trustworthiness criteria, JETERAPS, 5, 272
- Author, 2024, JCOM
- Author. The 75th Annual International Communication Association (ICA) Conference; 2025.
- Author, 2024, J Res Sci Teach
Cited by
Sorted by citations
Klein-Avraham, I., & Baram‐Tsabari, A. (2026) · Public Understanding of Science · doi:10.1177/09636625261442388 · Cited by 4
Self-citationThe framework of prior knowledge in online credibility evaluation
Kiili, C., Bråten, I., & Coiro, J. (2026) · Educational Psychologist · doi:10.1080/00461520.2026.2717081 · Cited by 1
Dabran‐Zivan, S., Klein-Avraham, I., & Baram‐Tsabari, A. (2026) · PLoS ONE · doi:10.1371/journal.pone.0348712
Self-citationThe Summary Comes First: How AI Overviews Reshape Consumer Product SearchPreprint
Angelis, M. D., Mazzù, M. F., & Sabatini, N. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7397080
Sabariah, S., Syahputri, I., Affryyana Febbryyana, L., Salsabilla, S., Hasanah, N., Ashari, A., & Halipah, S. (2026) · Nizamiyah Jurnal Sains Sosial dan Multidisiplin · doi:10.64691/w116zb07
Metadata
- Authors
- Dabran-Zivan, S., Klein-Avraham, I., & Baram-Tsabari, A.
- Affiliations
- Technion – Israel Institute of Technology (IL)
- Venue
- PLOS One
- Publisher
- Public Library of Science (PLoS)
- Published
- 2026-03-30
- Volume
- 21
- Issue
- 3
- Pages
- e0345300
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC13035123
- OpenAlex
- W7143322068
- Cited by
- 6
- References
- 44
- Citations by year
- 2026: 6
- FWCI
- 34.00
- Type
- journal article
- Language
- en
- Update notices
- correction (2026-04-30)
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Niedersächsisches Ministerium für Wissenschaft und Kultur (11-76251-2345/2021 (ZN 3854)])
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Educational Strategies and Epistemologies
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- generative AI, AI-generated content, source evaluation, corroboration, machine heuristic, chatbots, critical evaluation skills, search engines
- MeSH terms
- Intelligent Systems, Generative Artificial Intelligence, Adult, Artificial Intelligence, Female, Humans, Male
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
