College students' credibility assessments of GenAI‐generated information for academic tasks: An interview study

One common case of factual errors reported in academic writing is inaccurate or fabricated references.

“One common case of factual errors reported in academic writing is inaccurate or fabricated references (Athaluri et al., 2023;Emsley, 2023).”

• cites Athaluri et al., 2023 sourcePartly supported

  The Results report inaccurate DOIs and apparently fabricated references among 178 references in ChatGPT-generated research proposals; the Discussion describes 28 articles as products of AI hallucination. This supports the example of inaccurate or fabricated references, but the study does not establish how common such errors are in academic writing generally. (LLM, conf=0.85)

  Experimental — accuracy not yet measured
• cites Emsley, 2023 sourceSupported

Traditional web credibility cues, such as presence of sources, voice tone, length, information presentation, and interface aesthetics, were useful in assessing the credibility of GenAI-generated information.

“Traditional web credibility cues, such as the presence of sources, voice tone, length, information presentation, and interface aesthetics, were useful in assessing the credibility of GenAI-generated information (Choi & Stvilia, 2015;Sun et al., 2019).”

• cites Choi & Stvilia, 2015 sourceNot checked — no full text of the source
• cites Sun et al., 2019 sourcePartly supported

  The Results sections on Content and Design (page not provided) identify references, writing and language, amount and presentation of information, and interface appearance as cues consumers use to judge online health information. The paper does not establish that these cues were useful for assessing GenAI-generated information; that application goes beyond the source. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

Prior knowledge or familiarity with subject matter or task requirements influenced when and how students examined GenAI credibility.

“Prior knowledge of or familiarity with specific subject matter or the requirements of a particular task influenced when and how students examined GenAI credibility.”

For tasks involving factual or logical solutions, students checked statements of fact included in GenAI responses.

“For a task that involved factual or logical solutions, students checked statements of fact included in GenAI responses.”

The analysis identified five major heuristics used by students to evaluate GenAI-generated information: endorsement, intersystem consistency, self-confirmation, expectancy violation, and intrasystem consistency.

“Our analysis identified five major heuristics-endorsement, intersystem consistency, self-confirmation, expectancy violation, and intrasystem consistency-and students' actions involved in applying the heuristics to evaluate GenAI-generated information, as shown in Table 4.”

Students' trust in GenAI was an individual factor associated with motivation for credibility evaluation.

“Students' trust in GenAI was an individual factor associated with motivation for credibility evaluation, consistent with literature indicating that students' trust in a system or source may influence their evaluation behaviors (Rieh & Danielson, 2007).”

• cites Rieh & Danielson, 2007 sourceNot checked — no full text of the source

Task salience influenced students' motivation to evaluate GenAI credibility.

“Most participants considered academic tasks, such as homework assignments and research activities (e.g., literature review), critical enough to warrant spending time reviewing and evaluating the information from GenAI tools.”

A common strategy for GenAI credibility assessment was checking if the information matched other sources.

“One common strategy for GenAI credibility assessments mentioned by our participants was checking if the information matched other sources.”

When interacting with GenAI on unfamiliar subjects, students highlighted the inherent uncertainty of relying on GenAI-generated content.

“When interacting with GenAI tools on unfamiliar subjects, students highlighted the inherent uncertainty of relying on GenAI-generated content.”

Students inspected the compliance of organization and format of content with a given academic standard (e.g., APA Style, instructor guidelines).

“Students inspected the compliance of the organization and format of the content with a given academic standard (e.g., APA Style, instructor guidelines).”

Students conceptualized credibility using correctness-related terms such as accurate, factual, and right, especially in the context of erroneous outputs from GenAI tools.

“Students conceptualized credibility using correctnessrelated terms, such as accurate, factual, and right, especially in the context of erroneous outputs from GenAI tools, such as wrong answers to math problems and fabricated references for academic writing.”

Students also used human sources such as experts, acquaintances, or peers to assess GenAI credibility.

“Students also used human sources, such as experts on given topics (e.g., their professors), acquaintances they believed to be better equipped with the topic under investigation, or peers working on the same type of tasks.”

The most frequently used source in this triangulation approach was web search engines, primarily Google.

“The most frequently used source in this triangulation approach was web search engines, primarily Google”

In the study sample, low-salience tasks typically involved casual or personal purposes.

“In our study sample, such low-salience tasks typically involved casual or personal purposes”

Students used ChatGPT and other GenAI tools for three major work tasks: writing, programming, and learning.

“We identified that students used ChatGPT and other GenAI tools for three major work tasks: writing, programming, and learning.”

Some participants realized that assessing response quality could be time-consuming, requiring more effort than initially expected.

“Some participants realized that assessing response quality could be timeconsuming, requiring more effort than initially expected.”

College students used GenAI tools for academic tasks including writing, programming, learning, presenting, and teaching.

“The study identified various types of academic tasks for which students used ChatGPT, including writing, programming, and learning.”

• Also stated in: § 6.2 | Use of GenAI tools for academic tasks · plain

  “Students reported using GenAI tools to support different academic tasks, mainly involving writing, programming, learning, presenting, and teaching.”

Students considered the accuracy of references as a cue signaling information credibility.

“Beyond the mere presence of a source, students also considered the accuracy of references as a cue signaling information credibility.”

Students employing the self-confirmation heuristic made immediate judgments of credibility when the topic was familiar to them.

“Students employing the self-confirmation heuristic made immediate judgments of the credibility of GenAIgenerated information when the topic was familiar to them”

The analysis identified four factors influencing students' motivation to evaluate GenAI credibility in the academic context.

“Our analysis identified four factors influencing students' motivation and two related to their ability to evaluate GenAI credibility in the academic context (Table 2).”

The analysis identified nine constructs used by students as criteria for evaluating GenAI credibility: correctness, pertinence, creativity, refinedness, structural accuracy, currency, unbiasedness, explainability, and human-likeness.

“Our analysis identified nine constructs: correctness, pertinence, creativity, refinedness, structural accuracy, currency, unbiasedness, explainability, and human-likeness.”

When students considered a task to have low salience, they did not review and evaluate the content.

“Conversely, when students considered a task to have low salience, they did not review and evaluate the content.”

For problem-solving tasks with predetermined or correct answers, students tested ChatGPT's suggested answers using external tools such as a calculator or coding validator.

“For problem-solving tasks, which usually have predetermined or correct answers, students tested the answers suggested by ChatGPT using external tools, such as a calculator for a math problem or coding validator (e.g., W3School) for a coding problem.”

The analysis identified two factors related to students' ability to evaluate GenAI credibility in the academic context.

“Our analysis identified four factors influencing students' motivation and two related to their ability to evaluate GenAI credibility in the academic context (Table 2).”

Students conceptualized credibility differently when using GenAI tools for academic tasks.

“Students conceptualized credibility differently when using GenAI tools to complete academic tasks.”

Students working on tasks with specific guidelines, such as homework assignments, checked the credibility of GenAI-generated content against the grading criteria set by their instructors.

“When working on a task with specific guidelines, such as homework assignments, students checked the credibility of GenAIgenerated content against the grading criteria set by their instructors.”

Several students used Google to cross-check facts across multiple sources.

“Several students used Google to cross-check facts across multiple sources.”

Students aware of the pretrained nature of ChatGPT paid attention to the currency of its information.

“Those aware of the pretrained nature of ChatGPT paid attention to the currency of its information.”

Some students considered how natural and fluent the language used by LLM-based GenAI chatbots was as an important benchmark.

“Some students highlighted that LLM-based GenAI chatbots are designed and trained to mimic human language.Therefore, how natural and fluent the language the chatbots use was considered an important benchmark.”

The 10 cues clustered into three categories: application, content, and design.

“These cues clustered into three categories: application, content, and design (see Table 5).”

Academic tasks with GenAI involved fine-grained actions such as summarizing texts, generating new text or code, editing text, debugging code, and searching for information.

“These academic tasks involved more fine-grained actions, such as summarizing texts, generating new text or code, editing text, debugging code, and searching for information.”

Several students employed an iterative testing approach to validate whether the tool produced consistent content across multiple trials using similar prompts (intrasystem consistency heuristic).

“Several students employed an iterative testing approach to validate whether the tool produced consistent content across multiple trials using similar prompts.”

When highly motivated, students cross-checked information using available sources.

“When students were highly motivated, they cross-checked the information using available sources.”

Some students valued ChatGPT's explanations, particularly for problem-solving tasks, that articulated the reasoning or sourcing behind an output.

“Some students valued ChatGPT's explanations, particularly for problem-solving tasks, that articulated the reasoning or sourcing behind an output it provides”

For programming tasks, specific pieces of code were used as cues to judge the validity of GenAI solutions.

“For programming tasks, specific pieces of code were used as cues to judge the validity of the solutions suggested by GenAI tools.”

Students who mentioned refinedness expected GenAI to integrate information from relevant sources and process it to answer questions directly.

“Those who mentioned refinedness or related terms, such as concise, straightforward, and sophisticated, expected GenAI to integrate information from relevant sources and process it to answer their questions directly.”

Students prioritized evaluating GenAI-generated information for academic tasks, such as assignments and research activities, because they perceived the consequences of adopting the information (task salience) as high.

“Students in our study prioritized evaluating GenAI-generated information for academic tasks, such as assignments and research activities, given that they perceived the consequences of adopting the information (i.e., task salience) as high.”

Several participants viewed the presence of sources as an important cue in evaluating the credibility of GenAI responses.

“Several participants highlighted the significance of sources.They viewed the presence of sources as an important cue in evaluating the credibility of GenAI responses.”

The possibility of checking whether information was correct influenced students' willingness to evaluate information, especially for solution-driven (e.g., coding) or fact-checking tasks.

“The possibility of checking whether the given information was correct influenced students' willingness to evaluate information, especially for solution-driven (e.g., coding) or fact-checking tasks, for which immediate feedback is applicable and useful in credibility judgments (Metzger et al., 2003).”

• cites Metzger et al., 2003 sourceNot checked — no full text of the source

The study identified a new specific task, ideation, where students used ChatGPT as a starting point to develop project ideas.

“Nevertheless, we identified a new specific task, ideation, wherein students used ChatGPT as a starting point to develop project ideas.”

Social pressure from professors and cohort members influenced students' motivation to evaluate GenAI credibility.

“Students' social contacts in the academic realm, such as their professors and cohort members, appeared to influence their motivation to evaluate GenAI credibility.”

Creativity was considered mainly in creative writing tasks.

“Creativity was considered mainly in creative writing tasks.”

Some students mentioned the length of answers and favored conciseness.

“Some students mentioned the length of answers and favored conciseness.”

Pertinence was mentioned as a critical criterion for judging information credibility in problem-solving tasks to find relevant and applicable solutions in a particular context.

“Pertinence was mentioned as a critical criterion for judging information credibility in problem-solving tasks to find relevant and applicable solutions in a particular context.”

Students showed different levels of trust toward GenAI.

“Students showed different levels of trust toward GenAI.”

Highly knowledgeable users tended to evaluate the validity of content rather than systematically verifying it.

“Additionally, highly knowledgeable users tended to evaluate the validity of the content rather than systematically verifying it (i.e., self-confirmation heuristics; see more about this evaluation approach in the Heuristics for GenAI Credibility Assessments section).”

Time availability influenced students' approach to information evaluation.

“Several participants pointed out that time availability influenced their approach to information evaluation.”

Content structural elements such as grammatical accuracy and writing style of ChatGPT's responses were considered in credibility assessments.

“Content structural elements, such as the grammatical accuracy and writing style of ChatGPT's responses, were considered in credibility assessments.”

Students were less inclined to assess the credibility of GenAI information for creative tasks such as writing a poem.

“However, students were less inclined to assess the credibility of information for creative tasks, such as writing a poem.”

Social pressure, such as professors' instructions or policies regarding GenAI use for coursework, can affect students' motivation to assess information credibility.

“Social pressure, such as professors' instructions or policies regarding the use of GenAI tools for coursework, can affect students' motivation to assess information credibility.”

Task verifiability was an important motivational factor influencing students' willingness to evaluate information.

“Also, our study identified task verifiability as an important motivational factor.”

The study identified 10 cues of information credibility in the GenAI context: version, references, facts, codes, clichés, tone, length of information, presentation of information, interface type, and interface aesthetics.

“In the present study, we identified 10 cues: version, references, facts, codes, clichés, tone, length of information, presentation of information, interface type, and interface aesthetics.”

On ChatGPT, due to its minimalistic and familiar interface as a chatbot, design elements did not seem to affect students' perceptions of information credibility as much.

“On ChatGPT, however, due to its minimalistic and familiar interface as a chatbot, design elements did not seem to affect students' perceptions of information credibility as much.”

The study identified the structure and key factors influencing students' decision making on when and how deeply to engage in the credibility evaluation process.

“Guided by two existing models of credibility, we identified the structure and key factors influencing students' decision making on when and how deeply to engage in the credibility evaluation process.”

One unique cue found involved the version of the LLM used in the GenAI tool (e.g., GPT-3.5).

“One unique cue we found involved the version of the LLM used in the GenAI tool (e.g., GPT-3.5).”

Domain knowledge was associated with familiarity with tasks and skills and enabled students to evaluate information or solutions provided by ChatGPT.

“because domain knowledge was associated with their familiarity with tasks and skills and enabled them to evaluate information or solutions provided by ChatGPT.”

The only cue related to the application mentioned was its version.

“The only cue related to the application mentioned was its version.”

These factors included task-related, social, individual, and situational factors.

“These factors included task-related, social, individual, and situational factors (Table 2).”

Time availability emerged as a situational factor affecting students' capacity to assess the credibility of GenAI-generated information.

“Time availability emerged as a situational factor affecting students' capacity to assess the credibility of GenAI-generated information.”

Students' domain knowledge was an individual factor that influenced their ability to evaluate GenAI-generated information critically.

“Students' domain knowledge is another individual factor that influenced their ability to evaluate GenAIgenerated information critically, because domain knowledge was associated with their familiarity with tasks and skills and enabled them to evaluate information or solutions provided by ChatGPT.”

When students prioritize speed and convenience over information credibility, such as when facing tight assignment deadlines, they often skip or opt for quick, superficial evaluations.

“When students prioritize speed and convenience over information credibility, such as when facing tight assignment deadlines, they often skip or opt for quick, superficial evaluations, as reported in prior research (Metzger, 2007;Rieh & Hilligoss, 2008).”

• cites Metzger, 2007 sourceNot checked — no full text of the source
• cites Rieh & Hilligoss, 2008Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Participants reported using GenAI tools, primarily ChatGPT 3.5, to complete academic tasks, including writing, programming, and learning.

“The participants reported using GenAI tools, primarily ChatGPT 3.5, to complete academic tasks, including writing, programming, and learning.”

Students expressed appreciation for insights into why and how a certain answer was generated, especially for solution-driven tasks such as debugging (explainability).

“Moreover, students expressed appreciation for insights into why and how a certain answer was generated, especially for solution-driven tasks such as debugging (i.e., explainability; Shin, 2023).”

• cites Shin, 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Among the nine constructs, three were unique to GenAI systems: refinedness, human-likeness, and explainability.

“Among the nine constructs, three were unique to GenAI systems, reflecting evolving user expectations in the GenAI context: refinedness, human-likeness, and explainability.”

The newly added heuristic, intrasystem consistency, seems useful in evaluating GenAI credibility, given the probabilistic content-generation behavior of GenAI tools.

“The newly added heuristic, intrasystem consistency, seems useful in evaluating GenAI credibility, given the probabilistic contentgeneration behavior of GenAI tools (Peres et al., 2023).”

• cites Peres et al., 2023 sourceNot checked — no full text of the source

Lim's (2013) study found a significant positive correlation between professors' endorsement and students' perceptions of Wikipedia's credibility when Wikipedia was widely contested as a reliable source.

“This finding resonates with Lim's (2013) study, which found a significant positive correlation between professors' endorsement and students' perceptions of Wikipedia's credibility when Wikipedia was widely contested as a reliable source of information (e.g., Denning et al., 2005).”

• cites Lim, 2013 sourceNot checked — no full text of the source

The lack of understanding of the working mechanisms of LLMs may explain why the only application cue mentioned was version.

“This may be due to the lack of understanding of the working mechanisms of LLMs, especially on which data they were trained and by which rules they generate responses.”

On ChatGPT or similar GenAI-enabled systems, the version number or model iteration can serve as a cue indicating the currency of the training dataset, which is related to answer quality.

“However, on ChatGPT or other similar GenAI-enabled systems, the version number or model iteration can serve as a cue indicating the currency of the training dataset, which is related to answer quality.”

The study did not capture students' credibility evaluation strategies and tactics in relation to newer features (e.g., connection between generated content and sources).

“Our study did not capture students' credibility evaluation strategies and tactics in relation to these features.”

• AssumptionRests on an assumption: Current GenAI apps, such as ChatGPT-4 and Copilot, can offer a connection between generated content and sources and thus can facilitate source-based credibility evaluation.

Of the five heuristics identified, four (endorsement, intersystem consistency, self-confirmation, expectancy violation) aligned with those in the web credibility literature.

“Of the five heuristics we identified, four aligned with those in the web credibility literature (Hilligoss & Rieh, 2008;Metzger & Flanagin, 2013)-endorsement heuristic, intersystem consistency heuristic, self-confirmation heuristic, and expectancy violation heuristic.”

• cites Hilligoss & Rieh, 2008 sourceNot checked — no full text of the source
• cites Metzger & Flanagin, 2013 sourceNot checked — no full text of the source

The tasks that students mentioned during interviews were limited to academic tasks, so findings regarding motivational factors may not fully capture students' behaviors in assessing GenAI credibility in broader or nonacademic settings.

“Second, the tasks that students mentioned during the interviews were limited to academic tasks.Therefore, our findings regarding motivational factors may not fully capture students' behaviors in assessing GenAI credibility in broader or nonacademic settings.”