CoChat
Sign Up
Cited by 430Artificial Intelligence in Healthcare and EducationMachine Learning in HealthcareTopic Modeling

Almanac — Retrieval-Augmented Language Models for Clinical Medicine

Open access
Zakka, Cyril, Shad, Rohan, Chaurasia, Akash, Dalal, Alex R., Kim, Jennifer L., Moor, Michael, Fong, Robyn, Phillips, Curran +14 more·2024·NEJM AI
DOI 10.1056/aioa2300068PMC PMC10857783
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Large language models (LLMs) have recently shown impressive zero-shot capabilities, whereby they can use auxiliary data, without the availability of task-specific training examples, to complete a variety of natural language tasks, such as summarization, dialogue generation, and question answering. However, despite many promising applications of LLMs in clinical medicine, adoption of these models has been limited by their tendency to generate incorrect and sometimes even harmful statements. METHODS: We tasked a panel of eight board-certified clinicians and two health care practitioners with evaluating Almanac, an LLM framework augmented with retrieval capabilities from curated medical resources for medical guideline and treatment recommendations. The panel compared responses from Almanac and standard LLMs (ChatGPT-4, Bing, and Bard) versus a novel data set of 314 clinical questions spanning nine medical specialties. RESULTS: Almanac showed a significant improvement in performance compared with the standard LLMs across axes of factuality, completeness, user preference, and adversarial safety. CONCLUSIONS: Our results show the potential for LLMs with access to domain-specific corpora to be effective in clinical decision-making. The findings also underscore the importance of carefully testing LLMs before deployment to mitigate their shortcomings. (Funded by the National Institutes of Health, National Heart, Lung, and Blood Institute.).

AI snapshot

Written from the paper’s full text.

TL;DR

Almanac is a retrieval-augmented LLM framework for open-ended clinical question answering that grounds responses in curated sources and provides citations. In a clinician-graded evaluation on the novel ClinicalQA benchmark, Almanac outperformed ChatGPT-4, Bing, and Bard on factuality, completeness, preference, and adversarial safety.

Key findings

  • Across ClinicalQA, Almanac had the best mean ranks (lower is better): factuality 1.96±0.06, completeness 1.85±0.06, and preference 1.87±0.06, with significant differences from counterparts by Nemenyi testing (P<0.01); Bard performed significantly worse.
  • Almanac provided correct and trustworthy citations for 91% of ClinicalQA questions, compared with 82% for Bing; ChatGPT-4 citations were mostly nonexistent or unrelated, and Bard either relied on intrinsic knowledge or refused to cite sources.
  • In adversarial prompting, Almanac scored 100% correct versus Bard 77%, Bing 70%, and ChatGPT-4 7%. For errors of input, ChatGPT-4 and Almanac caught missing inputs, whereas Bard and Bing refused to perform clinical calculations.
  • On the LiveQA test set of 102 questions, Almanac obtained an average score of 2.85, versus the best-performing model reported in the associated publication at 0.637, described as a 347% improvement.

Methods

  • Almanac combines a vector database (Qdrant v1.3.0) initialized with more than 500 clinical calculators from MDCalc, a browser limited to PubMed, UpToDate, and BMJ Best Practices, a retriever using OpenAI text-embedding-ada-002 with threshold λ=0.83 and n=10, and gpt-4-0613 with an 8192-token maximum. Articles were chunked into 1000-token passages.
  • The evaluation used ClinicalQA, a novel benchmark of 314 open-ended clinical questions generated by a panel of eight board-certified clinicians and two health care practitioners with an average of 10.5 years of experience across nine medical specialties. A subset of 25 questions was used for adversarial prompting, and five clinical calculation vignettes were used for errors of input.
  • Clinician panelists independently ranked outputs from Almanac, ChatGPT-4, Bing, and Bard on factuality, completeness, and preference. Statistical significance was assessed with the Friedman test at P<0.01 followed by Nemenyi post hoc pairwise comparisons; citation quality was scored binary, and LiveQA responses were manually scored 0–3.

Limitations

  • The paper states that Almanac has limitations in ranking information sources by criteria such as evidence level, study type, and publication date; it uses general-purpose retrievers and language models that could be fine-tuned; ClinicalQA metrics are subjective and rely on human graders, posing scalability challenges; and integration into health care should be cautious because of flaws in generating accurate responses and handling questions lacking straightforward answers.
  • The paper also notes that off-the-shelf adversarial resilience becomes less effective as adversarial prompts decrease in word count, requiring careful λ tuning to balance true- and false-positive findings, and that Almanac still struggled with specific resources such as providing a clinical or physician recommendation.
  • Evident from the text: complete blinding was not possible because of differing prose styles; alternate retrieval strategies were not explored; adversarial evaluation used only a subset of 25 ClinicalQA questions; and all evaluations were conducted before July 30, 2023, with specific model versions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1056/aioa2300068

Cites

62 of 63 references verified1 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 25 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    For Almanac, addition of the adversarial prompt lowered the average score between the query and the retrieved articles below threshold λ, resulting in the system abstaining from responding to a given prompt.

    “We note that for Almanac, the addition of the adversarial prompt lowered the average score between the query and the retrieved articles below the threshold (λ), resulting in the system abstaining from responding to a given prompt.”
    Quote in the paperSubstantial claim
  • Findingown data

    The framework achieves significant improvements in preference compared with baselines as assessed by a panel of board-certified physicians and health care workers.

    “We evaluated our approach on a novel data set of clinical questions and showed that our framework achieves significant improvements in factuality, completeness, preference, and adversarial safety compared with baselines as assessed by a panel of board-certified physicians and health care workers.”
    Quote in the paperSubstantial claim
  • Findingown data

    Almanac's adversarial safety performance greatly superseded Bard, Bing, and ChatGPT-4 in adversarial prompting (100% vs 77%, 70%, and 7%, respectively).

    “Regarding adversarial safety, Almanac's performance greatly superseded that of Bard, Bing, and ChatGPT-4 in adversarial prompting (100% vs. 77, 70, and 7%, respectively).”
    Quote in the paperSubstantial claim
  • Findingown data

    Nemenyi P values (P<0.01) showed significant differences between Almanac and its counterparts.

    “These results were echoed by the Nemenyi P values (P<0.01) in Figure 2 showing significant differences between Almanac and its counterparts.”
    Quote in the paperSubstantial claim
  • Findingnone given

    Almanac still struggled with specific resources, such as providing a clinical or physician recommendation.

    “We note that although Almanac was able to provide more up-to-date answers than the ones provided at times, it still struggled with specific resources, such as providing a clinical or physician recommendation.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 15
  • May combine two claims. 24

  • Classified as probably not a substantial claim. May combine two claims.

    Findingnone given

    Grounded language models, such as Almanac, are not without flaws, particularly in generating accurate responses and handling questions that lack straightforward answers in their data sources.

    “They are not without flaws, however, particularly in generating accurate responses and handling questions that lack straightforward answers in their data sources.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On LiveQA, Almanac obtained an average score of 2.85 across 102 questions, well above the best-performing model reported in the article with an average score of 0.637 (a 347% improvement).

    “On LiveQA, Almanac obtained an average score of 2.85 across the 102 questions, well above the best-performing model reported in the article with an average score of 0.637 (a 347% improvement).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Similar trends were observed for the weighted mean and inverse-weighted mean, with slightly better weighted mean performance by ChatGPT-4 in anesthesia (factuality, 5.23±0.15 vs. 5.60±0.22) and matched performances in cardiology (factuality, 6.20±0.11 vs. 6.20±0.14).

    “Similar trends were observed for the weighted mean (with lower ranks penalized more heavily) and inverse-weighted mean (with higher ranks weighed more heavily), with a noted slightly better weighted mean performance by ChatGPT-4 in anesthesia (factuality, 5.23±0.15 vs. 5.60±0.22)and matched performances in cardiology (factuality, 6.20±0.11vs. 6.20±0.14).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Galactica 120B, RAG, and BioMedLM had very poor performance, so only Almanac, ChatGPT-4, Bing, and Bard were used for further evaluation.

    “we fed each question into a series of models: Almanac, ChatGPT-4 (May 24, 2023 version), Bing (June 28, 2023), Bard (June 28, 2023), Galactica 120B, RAG, and BioMedLM (formerly known as PubMed GPT).We then retrieved their answers.Because of the very poor performance by the last three models (Supplementary Appendix), only Almanac, ChatGPT-4, Bing, and Bard were used for further evaluation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Bard either relied on its intrinsic knowledge or refused to cite sources despite being prompted to do so.

    “Although ChatGPT-4 citations were mostly plagued by nonexistent or unrelated web pages, Bard either relied on its intrinsic knowledge or refused to cite sources, despite being prompted to do so.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Almanac was able to provide more up-to-date answers than the ones provided at times.

    “We note that although Almanac was able to provide more up-to-date answers than the ones provided at times, it still struggled with specific resources, such as providing a clinical or physician recommendation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Despite slightly worse results in the performance of Bing compared with ChatGPT-4 on ClinicalQA, the Nemenyi heat maps showed no significant differences in their performances.

    “Despite slightly worse results in the performance of Bing compared with ChatGPT-4 on the ClinicalQA data set, the Nemenyi heat maps showed no significant differences in their performances.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Both ChatGPT-4 and Almanac were able to catch missing inputs.

    “For errors of input, although Bard and Bing refused to perform any clinical calculations, both ChatGPT-4 and Almanac were able to catch missing inputs (Supplementary Appendix).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Bing achieved a citation performance of 82% on ClinicalQA because of unreliable sources, including personal blogs and online forums, despite providing sources for every question.

    “However, despite providing sources for every question on the data set, Bing achieved a performance of 82% because of unreliable sources, including personal blogs and online forums.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Bing complied or refused to respond altogether in adversarial prompting.

    “Interestingly, Bard was also able to output the correct response along with the adversarial output despite being prompted not to, whereas Bing complied or refused to respond altogether.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Across the full scope of ClinicalQA, Almanac outperformed its counterparts, with a mean rank of 1.96±0.06 in factuality, 1.85±0.06 in completeness, and 1.87±0.06 in preference, with similar overall performances within each specialty.

    “Across the full scope of ClinicalQA, Almanac outperformed its counterparts, with a mean rank of 1.96±0.06 in factuality, 1.85±0.06 in completeness, and 1.87±0.06 in preference, with similar overall performances observed within each specialty (Table 3 and Supplementary Appendix).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Using data sets with public-facing evaluations hinders objective experimentation, thereby undermining any subsequently drawn conclusions.

    “As such, using data sets with public-facing evaluations hinders objective experimentation, thereby undermining any subsequently drawn conclusions.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The training data for many proprietary models are often kept confidential, introducing an extra layer of uncertainty in estimating data contamination.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    Graders were made aware of the study's goals, but they were not cognizant of the models being tested or familiar with the prose styles adopted by each model.

    “Graders were made aware of the study's goals, but they were not cognizant of the models being tested or familiar with the prose styles adopted by each model.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning

    Because these systems leverage knowledge encoded within their weights to answer clinical queries, their outputs become contingent on the assumption that correct information outweighs misinformation within their training data set.

    “Moreover, because these systems leverage the knowledge encoded within their weights to answer clinical queries, their outputs become contingent on the assumption that correct information outweighs misinformation within their training data set.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductnone given

    The evaluation metrics are subjective and rely on human graders, posing challenges for scalability.

    “It is also worth noting that although our ClinicalQA benchmark shows promise, the evaluation metrics are subjective and rely on human graders, posing challenges for scalability.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Prompting models with the standard 'think carefully and step by step' before each question has been shown to generally lead to more accurate outputs.

    “The latter models were prompted with the standard”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [56]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Large language models can be prone to generating incorrect statements, known as hallucinations, which limits their adoption in clinical medicine.

    “However, despite many promising applications of LLMs in clinical medicine, adoption of these models has been limited by their tendency to generate incorrect and sometimes even harmful statements.”
    • Also stated in: § Background · plain
      “However, because of the nature of the training objective of predicting the next token in a sentence, large language models (LLMs) can be prone to generating incorrect statements, a phenomenon known as hallucinations.8,9”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [8]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [9]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Evaluators were explicitly instructed to distinguish between the performance of different models rather than assigning them equivalent scores.

    “The evaluators were explicitly instructed to distinguish between the performance of different models rather than assigning them equivalent scores.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Physicians often make use of point-of-care tools to improve outcomes.

    “To stay abreast of the shifting landscape of evidence-based practices, physicians often make use of point-of-care tools to improve outcomes.20”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [20]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    The websites used by the browser are carefully curated to ensure high-quality content in response to queries.

    “These websites are carefully curated to ensure high-quality content in response to queries.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Large language models may reproduce social biases and make statements that reinforce gender, racial, and religious stereotypes.

    “Moreover, studies have shown that these models may reproduce social biases and make statements that reinforce gender, racial, and religious stereotypes.10,11”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [10]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [11]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    U.S. Medical Licensing Examination-style questions fail to encapsulate the full scope of actual clinical scenarios; they portray patient scenarios as neat clinical vignettes, bypassing the intricate microdecisions of real patient care.

    “In addition to showing poor to weak positive clinical correlation, [43][44][45][46][47][48] U.S. Medical Licensing Examination-style questions fail to encapsulate the full scope of actual clinical scenarios encountered by medical professionals.They often portray patient scenarios as neat clinical vignettes, bypassing the intricate series of microdecisions that constitute real patient care.”
    Quote in the paperSubstantial claim0 of 6 sources checked
    • cites [43]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [44]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [45]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [46]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [47]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [48]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    With this framework, large language model (LLM) outputs remain grounded in truth while providing a reliable way of fact-checking.

    “With this framework, large language model (LLM) outputs remain grounded in truth while providing a reliable way of fact-checking.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The predefined sources used by Almanac are trustworthy.

  • May combine two claims.

    Premisereasoning

    Existing data sets such as MultiMedQA, MedMCQA, and PubMedQA serve as valid benchmarks for reading comprehension and knowledge recall, but their use in open-ended clinical QA poses problems of data contamination and poor clinical proxies.

    “Although existing data sets, such as MultiMedQA, 30 MedMCQA, 40 and PubMedQA, 41 serve as valid benchmarks for assessing reading comprehension and knowledge recall of biomedical language models, their use as benchmarks in open-ended clinical QA tasks poses two clear problems: data contamination and poor clinical proxies.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [30]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [40]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [41]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    By supplementing responses with passages from predefined sources, our grounded system is able to dampen explainability concerns by enabling clinicians to independently verify outputs from trustworthy sources, leading to significant improvements over general retrieval systems.

    “By supplementing responses with passages from predefined sources, our grounded system is able to dampen explainability concerns by enabling clinicians to independently verify outputs from trustworthy sources, leading to significant improvements over general retrieval systems.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The predefined sources used by Almanac are trustworthy.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    Efforts were made to ensure unbiased grading by arbitrary answer formatting and answer order shuffling to blind physicians to the answer's provenance, but complete blinding was not possible because of the prose styles adopted by each system.

    “Although efforts were made to ensure unbiased grading (e.g., arbitrary answer formatting, answer order shuffling) to blind physicians to the answer's provenance, complete blinding was not possible because of the prose styles adopted by each system.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The value of the λ=0.83 retrieval threshold was based on an analysis of retriever scores for a sample of 25 clinical questions from an external data set.

    “This threshold was on the basis of an analysis of retriever scores for a sample of 25 clinical questions from an external data set.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Despite strong performances on medical question-answer data sets such as MedQA, these models failed to translate to real-world clinical scenarios because of potential harm from unmitigated hallucinations, benchmarks that do not accurately reflect clinically relevant tasks, and concerns of data contamination between train-test splits.

    “Despite strong performances on medical question-answer data sets, such as MedQA, 59 these models failed to translate to real-world clinical scenarios because of the potential for harm from unmitigated hallucinations, benchmarks that do not accurately reflect clinically relevant tasks, and concerns of data contamination between train-test splits.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [59]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    There was no overlap between graders, as all were asked to evaluate their own submitted questions in accordance with their specialties and fellowships.

    “There was no overlap between graders, as all were asked to evaluate their own submitted questions in accordance with their specialties and fellowships.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Because of the considerable burden and information overload experienced by health care workers, it is not uncommon for medical errors to occur because of human error and poor documentation.

    “Because of the considerable burden and information overload experienced by health care workers in their day-to-day responsibilities, it is not uncommon for medical errors to occur because of human error and poor documentation.43”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [43]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Each retrieved passage is processed independently of other chunks, and alternate retrieval strategies were not explored.

    “We noted that each retrieved passage is processed independently of other chunks, and alternate retrieval strategies were not explored.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Current LLM evaluation metrics rely on automated methods such as bilingual evaluation understudy, but they fail to fully capture the complexity and nuances of medical retrieval tasks.

    “Current LLM evaluation metrics rely on automated methods, such as bilingual evaluation understudy, 55 but they fail to fully capture the complexity and nuances of medical retrieval tasks.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [55]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

32 of 32

  • The paper relies on this without support.

    As clinical evidence continues to grow, curated content becomes less accessible and more confined to error-prone search tools and time-consuming appraisal techniques that fail to address the unique needs of individual patients.

    “However, as clinical evidence continues to grow, curated content becomes less accessible and more confined to error-prone search tools and time-consuming appraisal techniques that fail to address the unique needs of individual patients.”

  • The paper relies on this without support.

    The training data for many proprietary models are often kept confidential, introducing an extra layer of uncertainty in estimating data contamination.

    “More concerning, the training data for many proprietary models are often kept confidential, introducing an extra layer of uncertainty in estimating data contamination.”

  • The paper relies on this without support.

    The predefined sources used by Almanac are trustworthy.

    “By supplementing responses with passages from predefined sources, our grounded system is able to dampen explainability concerns by enabling clinicians to independently verify outputs from trustworthy sources”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Zakka et al. (2024) demonstrated an 18% increase in factuality compared to ChatGPT using a web browser and an unspecified corpus of medical text and guidelines to pre-retrieve relevant information.

    “Implementations of RAG in the medical field include Zakka et al. (2024), who use a web browser along with an unspecified corpus of medical text and guidelines to pre-retrieve relevant information, which is then passed to a LLM. The system demonstrated an 18% increase in factuality compared to ChatGPT.”

    From Grounding Large Language Models in Clinical Evidence: A Retrieval-Augmented Generation System for Querying UK NICE Clinical Guidelines (2025) · cites this paper as Zakka et al. (2024)

    The Architecture section describes a browser that retrieves material for the language model, but it specifies the sources—PubMed, UpToDate, and BMJ Best Practice—rather than leaving the corpus unspecified. The Results section reports better factuality rankings for Almanac than for ChatGPT-4, but does not report an 18% increase in factuality; its reported measure is mean rank (1.96 versus 2.34, where lower is better). (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • One study used adversarial prompting to evaluate the safety of the RAG system.

    “One study used adversarial prompting, defined as intentionally adding harmful directives to a prompt, to evaluate the safety of the RAG system.28”

    From Improving large language model applications in biomedicine with retrieval-augmented generation: a systematic review, meta-analysis, and clinical development guidelines (2025) · cites this paper as [28]

  • Almanac uses RAG frameworks to convert clinical QA tasks into search and retrieval processes, which use LLMs for knowledge distillation from authoritative medical sources to minimize hallucination risks.

    “Almanac (Zakka et al., 2024) uses RAG frameworks to convert clinical QA tasks into search and retrieval processes, which use LLMs for knowledge distillation from authoritative medical sources to minimize hallucination risks.”

    From Trustworthy Medical Question Answering: An Evaluation-Centric Survey (2025) · cites this paper as Zakka et al., 2024

  • RAG temporarily enhances an LLM's knowledge by incorporating relevant text excerpts from authoritative sources into the model.

    “retrieval-augmented generation (RAG)15, a process that temporarily enhances an LLM’s knowledge by incorporating relevant text excerpts from authoritative sources into the model, such as medical guidelines16 or textbooks.”

    From Development and validation of an autonomous artificial intelligence agent for clinical decision-making in oncology (2025) · cites this paper as [16]

  • Recent datasets by Zakka et al. target open-ended responses and LongHealth focuses on patient-related content.

    “Recent advancements have been made with the introduction of new datasets by Zakka et al.16, targeting the enhancement of open-ended responses, and LongHealth27, focusing on patient-related content.”

    From Development and validation of an autonomous artificial intelligence agent for clinical decision-making in oncology (2025) · cites this paper as [16]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Below-threshold retrieval leads to both an answer and abstention

    Major

    The methods say Almanac answers from intrinsic knowledge when no retrieved articles exceed the threshold. The results say an adversarial prompt lowered retrieval scores below that threshold and caused Almanac to abstain instead.

  • LiveQA test-set citation points to a medical-exams dataset

    Minor

    The methods cite reference 57 for LiveQA, but reference 57 describes a question-answering dataset from medical exams. The LiveQA reference is numbered 55.

  • Step-by-step prompting citation points to a different topic

    Minor

    The text cites reference 56 for the claim about step-by-step prompting, but that reference concerns language-knowledge graph pretraining.

  • Browser-domain citations point to different sources

    Minor

    The browser section cites references 52-54 for PubMed, UpToDate, and BMJ Best Practices, respectively. Reference 52 is BMJ Best Practice, while references 53 and 54 are unrelated papers.

  • BLEU citation points to a LiveQA paper

    Minor

    The evaluation section cites reference 55 for bilingual evaluation understudy, but reference 55 is about the LiveQA task. The BLEU paper appears as reference 53.

5 of 5

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. Lewis P Perez E Piktus A et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. NIPS’20: Proceedings of the 34th International Conference on Neural Information Processing Systems December 2020 (https://proceedings.neurips.cc/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf).
  2. Borgeaud S Mensch A Hoffmann J et al. Improving language models by retrieving from trillions of tokens. International Conference on Machine Learning (ICML) 2022.
  3. Wolters Kluwer. UpToDate. 2023 (https://www.uptodate.com).
  4. BMJ Publishing Group. BMJ Best Practice. 2023 (http://bestpractice.bmj.com).

Cited by

Sorted by citations

Related

Authors
Zakka, C., Shad, R., Chaurasia, A., Dalal, A. R., Kim, J. L., Moor, M., Fong, R., Phillips, C., Alexander, K., Ashley, E., Boyd, J., Boyd, K., Hirsch, K., Langlotz, C., Lee, R., Melia, J., Nelson, J., Sallam, K., Tullis, S., Vogelsong, M. A., Cunningham, J. P., & Hiesinger, W.
Affiliations
  • Stanford Medicine (US)
  • University of Pennsylvania Health System (US)
  • Stanford University (US)
  • Department of Radiology and Biomedical Informatics, Stanford Medicine, Stanford, CA
  • Johns Hopkins University (US)
  • Columbia University (US)
Corresponding author
Zakka, Cyril
ORCID
Venue
NEJM AI
Publisher
Massachusetts Medical Society
Published
2024-01-25
Volume
1
Issue
2
ISSN
2836-9386 (ISSN-L)
DOI
10.1056/aioa2300068
PMC
PMC10857783
OpenAlex
W4391221150
Cited by
424
References
26
Citations by year
2026: 163 · 2025: 159 · 2024: 99 · 2023: 3
FWCI
39.53
Type
journal article
Language
en
Open access
Licence
other-oa
Locations
  • NEJM AI (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion, other-oa)
Funders
  • National Institutes of Health
  • National Heart, Lung, and Blood Institute (R01HL157235)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
retrieval-augmented language models, large language models, medical question answering, clinical guidelines, treatment guidelines, factuality, clinical decision support, almanacs

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review