The DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index; 14 of the 44 sources its claims cite were checked for support.
Evolving Health Information–Seeking Behavior in the Context of Google AI Overviews, ChatGPT, and Alexa: Interview Study Using the Think-Aloud Protocol
Open accessPublished version of a preprint
Abstract
We conducted in-depth qualitative research with 27 participants (ages 19-80 years) using a think-aloud protocol. Participants searched for health information across 3 platforms—Google, ChatGPT, and Alexa—while verbalizing their thought processes. Participants integrated AI tools into their broader search routines rather than using them in isolation. ChatGPT was valued for its clarity, speed, and ability to offer broad overviews, but participants often questioned its source transparency and accuracy.
Overview
CoChat Summary
TL;DR
This qualitative think-aloud interview study with 27 participants found that people do not simply “trust” or “reject” AI health tools; instead, they mix Google AI Overviews, ChatGPT, and Alexa into broader search routines depending on context, urgency, and perceived usefulness. Across platforms, participants valued speed and summarization but were wary of missing sources, bias, and limited or inaccessible responses, especially for more serious health questions.
Key findings
- Of 27 participants, 23 (85%) reported favorably on some aspect of ChatGPT’s formatting or efficiency, while 14/27 already used it as part of their search routine.
- Google’s AI Overviews were often treated cautiously: about one-third of participants commented on them, 5/27 explicitly skipped them, and no participant reported stopping their search at the overview.
- Alexa was seen as useful mainly for low-stakes tasks: 17/27 (63%) said they would not typically use it for the study questions, while 8/27 (30%) said short voice responses were convenient for simple questions.
- Utility often outweighed trust: 15/27 (56%) said they would still use AI-generated information if it helped direct further investigation, and participants frequently described AI tools as “starting points” rather than final authorities.
Methods
- Qualitative interview study using a think-aloud protocol with 27 participants aged 19-80 years, conducted in summer and fall 2024.
- Participants searched for two health prompts on three platforms: Google, ChatGPT 3.5, and Alexa; one prompt was a standardized St. John’s wort question and the other was a personally relevant health query.
- Transcripts (370 pages) were analyzed by the three researchers using reflexive thematic analysis, with 34 codes developed and organized around two research questions.
Limitations
- The paper states the sample was small, recruited through loose personal-network connections, and not longitudinal; it warns the findings cannot be generalized to a wider population.
- The think-aloud protocol may have produced a Hawthorne effect, with participants changing behavior because they knew they were being observed.
- The paper notes that the technologies changed during data collection, so findings may quickly become outdated; this is an evident limitation given platform updates and evolving AI Overviews/ChatGPT source-linking behavior.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
11 Valid · 2 Not valid · 17 Not checked · 109 Own results and interpretations
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“More recent perspectives discuss how trust is iterative in online health contexts, but AI tools seem to be making this overlap between exploration and evaluation central to the search process [42].”
§ Evolving Health Information–Seeking Behavior in the Context of Google AI Overviews, ChatGPT, and Alexa: Interview Study Using the Think-Aloud Protocol > Discussion > Principal Findings
What the sources say
- Could not verifycites [42] sourceExperimental — accuracy not yet measured
The Background and Principal Findings discuss trust judgments and corroboration during online health-information searches. I searched for AI tools, iterative trust, and overlap between exploration and evaluation, but found no passage addressing whether AI makes that overlap central to the search process. (LLM, conf=0.75)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Second, our use of the think-aloud protocol to observe search behavior has the potential for producing the Hawthorne effect [63], in which participants change their behavior because they know they are being observed.”
§ Evolving Health Information–Seeking Behavior in the Context of Google AI Overviews, ChatGPT, and Alexa: Interview Study Using the Think-Aloud Protocol > Discussion > Limitations This study had several important limitations. First, our sample was small and generated via loose connections in personal networks. Second, our use of the think-aloud protocol to observe search behavior has the potential for producing the Hawthorne effect [63], in which participants change their behavior because they know they are being observed. Finally, this study was not longitudinal. It provides a snapshot of experiences with AI- and voice-assisted technologies, with a small sample of people, at one particular point in time. Notably, 10 of the 27 (37%) participants had never previously used a stand-alone LLM; therefore, we were able to observe their first impressions of using ChatGPT rather than discuss existing experiences or attitudes. Unfortunately, undertaking research on these types of technologies means that results quickly become out of date. In the context of this research, the functionality of Google’s AI Overviews and ChatGPT changed during the short time frame of the study. Google’s AI Overviews now appear in response to more search queries and are longer in length compared to the period when the interviews were being conducted. Additionally, both Google Overviews and ChatGPT now link out to sources. A majority of the participants had noted the absence of sources in both tools as a serious drawback, which suggests that interviews conducted today would likely sound very different in relation to this issue. While we are describing this issue as a limitation of our research, we also, as a wider field, need to grapple with the challenges of conducting research on technologies that are evolving at such speed. Authors need to focus on which of their findings are likely to withstand platform design changes and model updates and which will not. More research is also needed to explore whether users understand these design changes.
What the sources say
- Could not verifycites [63] sourceExperimental — accuracy not yet measured
The Materials and methods section describes clinical scenarios entered into ChatGPT and graded by physicians; the Discussion addresses limitations of that evaluation. Neither discusses a think-aloud protocol, observed search behavior, or the Hawthorne effect (searched those terms and the claim's description of behavior changing under observation). Our copy therefore does not verify the point attributed to this source. (LLM, conf=0.76)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In total, 74% (n=20) of the sample had at least an undergraduate degree”
§ Evolving Health Information–Seeking Behavior in the Context of Google AI Overviews, ChatGPT, and Alexa: Interview Study Using the Think-Aloud Protocol > Methods > Recruitment
plainn = 20- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants also appreciated that the platform allowed them to ask questions more precisely and that the conversational flow of the platform was responsive to how they worded their searches:”
§ Evolving Health Information–Seeking Behavior in the Context of Google AI Overviews, ChatGPT, and Alexa: Interview Study Using the Think-Aloud Protocol > Results > Observations Specifically Related to the 3 Technologies
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“participants in this study selectively engaged with AI- and voice-assisted tools based on perceived usefulness, not just trustworthiness”
§ Evolving Health Information–Seeking Behavior in the Context of Google AI Overviews, ChatGPT, and Alexa: Interview Study Using the Think-Aloud Protocol > Abstract > Conclusions
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
76 more claims
Create a free account to see all 139 claims and to verify the 5 references still open.
- Probably not a substantial claim. 36
- It may state two separate claims as one. 29
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Exposure to ideologically diverse news and opinion on Facebook
Bakshy, E., Messing, S., & Adamic, L. A. (2015) · Science · doi:10.1126/science.aaa1160 · Cited by 3,245 · cited 1×
“There is significant research on the ways in which algorithms produce search results that mirror people’s worldviews, but the majority of that work has been carried out on social networks [83,84].”
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921
McCambridge, J., Witton, J., & Elbourne, D. R. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.08.015 · Cited by 2,748
The Persuasiveness of Source Credibility: A Critical Review of Five Decades' Evidence
Pornpitakpan, C. (2004) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.2004.tb02547.x · Cited by 2,205
Filter Bubbles, Echo Chambers, and Online News Consumption
Flaxman, S., Goel, S., & Rao, J. M. (2016) · Public Opinion Quarterly · doi:10.1093/poq/nfw006 · Cited by 1,992 · cited 1×
“There is significant research on the ways in which algorithms produce search results that mirror people’s worldviews, but the majority of that work has been carried out on social networks [83,84].”
Eysenbach, G. (2002) · BMJ · doi:10.1136/bmj.324.7337.573 · Cited by 1,915 · cited 1×
“For over a decade, surveys have demonstrated that search engines are the most common platform for online information seeking [2], and this is supported in studies with a range of people, from everyday users [3,4] to patients with chronic health conditions [5-7].”
Hesse, B. W., Nelson, D. E., Kreps, G. L., Croyle, R. T., Arora, N. K., Rimer, B. K., & Viswanath, K. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.22.2618 · Cited by 1,523 · cited 1×
“Existing models emphasize credibility, trustworthiness, and usefulness as primary drivers of HISB [75-77].”
Social and Heuristic Approaches to Credibility Evaluation Online
Metzger, M. J., Flanagin, A. J., & Medders, R. B. (2010) · Journal of Communication · doi:10.1111/j.1460-2466.2010.01488.x · Cited by 1,394
Models of ecological rationality: The recognition heuristic.
Goldstein, D. G., & Gigerenzer, G. (2002) · Psychological Review · doi:10.1037/0033-295x.109.1.75 · Cited by 1,352 · cited 1×
“Psychologists have described this phenomenon as the illusory truth effect, explaining that people are more likely to believe information they have heard before, even if that information is false [78-80].”
Research works printed but not linked: 3
- Hovland, CI, 1953, Communication and Persuasion; Psychological Studies of Opinion Change, xii
- Amazeen, MA, 2023, Int J Commun, 17, 23
- Oh, P, 2023, Research Handbook on Artificial Intelligence and Communication
Cited by
Trillo-Domínguez, M., Martín-Neira, J.-I., & Olvera‐Lobo, M. (2026) · Informatics · doi:10.3390/informatics13030041 · Cited by 4
“These hallucinations primarily manifest in the generation of fabricated references, in the absence of evidence or citations without a verifiable real origin [30,46–49].”
Tiao, P.-H., Chan, H., & Huang, Y. (2026) · Journal of Medical Internet Research · doi:10.2196/82081 · Cited by 5
“The integration of large language model–based chatbots represents a further transformation in OHIS [10,126].”
Alon, L., & Levkovich, I. I. (2026) · Journal of Medical Internet Research · doi:10.2196/93944 · Cited by 5
“In 5 studies [6,7,13,25,35], evaluation of outputs felt more manageable when tools provided concrete verification and reuse affordances, such as visible sourcing, accessible transcripts, and options to save, revisit, or share content.”
InfluentialAn Eye Tracking Study: Are AI Overviews Changing Search Behavior?
Lawati, S. F. D. A., McKay, D., Sanderson, M., Thomas, P., & Trippas, J. R. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809530 · Cited by 5
“A recent study by Wardle et al. [41] re-searched how people meet their health-related information needs via search engines, including Google’s “AI Overviews” (Powered by Gemini), ChatGPT, and Alexa.”
Scherenberg, V. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1891658 · Cited by 5
“These answers may appear complete and personally relevant, even though users often cannot immediately judge how accurate, current, evidence-based, or appropriate they are for their own situation (1, 3).”
Metadata
- Authors
- Wardle, C., Urbani, S., & Wang, E.
- Affiliations
- Cornell University (US)
- Brown University (US)
- Corresponding author
- Wardle, Claire
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2025-10-07
- Volume
- 27
- Pages
- e79961
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC12541266
- OpenAlex
- W4414280203
- Cited by
- 43
- References
- 94
- Citations by year
- 2026: 43
- FWCI
- 53.84
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Keywords
- health information seeking, AI Overviews, ChatGPT, voice assistants, Amazon Alexa, large language models, search behavior, trust in AI, credibility assessment, perceived usefulness, multilingual users, digital health literacy, platform selection, data provenance, voice recognition, cultural relevance, Google search results, user demographics
- MeSH terms
- Generative Artificial Intelligence, Adult, Aged, Aged, 80 and over, Artificial Intelligence, Female, Humans, Interviews as Topic, Male, Middle Aged, Internet, Qualitative Research, Young Adult, Information Seeking Behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evolving Health Information–Seeking Behavior in the Context of Google AI Overviews, ChatGPT, and Alexa: Interview Study Using the Think-Aloud Protocol. https://app.cochat.ai/papers/10.2196/79961
- The paper
- Wardle, C., Urbani, S., & Wang, E. (2025). Evolving Health Information–Seeking Behavior in the Context of Google AI Overviews, ChatGPT, and Alexa: Interview Study Using the Think-Aloud Protocol. Journal of Medical Internet Research. https://doi.org/10.2196/79961
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.