Almanac — Retrieval-Augmented Language Models for Clinical Medicine

In the paper

“They are not without flaws, however, particularly in generating accurate responses and handling questions that lack straightforward answers in their data sources.”

In the paper

“On LiveQA, Almanac obtained an average score of 2.85 across the 102 questions, well above the best-performing model reported in the article with an average score of 0.637 (a 347% improvement).”

In the paper

“Similar trends were observed for the weighted mean (with lower ranks penalized more heavily) and inverse-weighted mean (with higher ranks weighed more heavily), with a noted slightly better weighted mean performance by ChatGPT-4 in anesthesia (factuality, 5.23±0.15 vs. 5.60±0.22)and matched performances in cardiology (factuality, 6.20±0.11vs. 6.20±0.14).”

In the paper

“we fed each question into a series of models: Almanac, ChatGPT-4 (May 24, 2023 version), Bing (June 28, 2023), Bard (June 28, 2023), Galactica 120B, RAG, and BioMedLM (formerly known as PubMed GPT).We then retrieved their answers.Because of the very poor performance by the last three models (Supplementary Appendix), only Almanac, ChatGPT-4, Bing, and Bard were used for further evaluation.”

In the paper

“Although ChatGPT-4 citations were mostly plagued by nonexistent or unrelated web pages, Bard either relied on its intrinsic knowledge or refused to cite sources, despite being prompted to do so.”

In the paper

“We note that although Almanac was able to provide more up-to-date answers than the ones provided at times, it still struggled with specific resources, such as providing a clinical or physician recommendation.”

In the paper

“Despite slightly worse results in the performance of Bing compared with ChatGPT-4 on the ClinicalQA data set, the Nemenyi heat maps showed no significant differences in their performances.”

In the paper

“For errors of input, although Bard and Bing refused to perform any clinical calculations, both ChatGPT-4 and Almanac were able to catch missing inputs (Supplementary Appendix).”

In the paper

“However, despite providing sources for every question on the data set, Bing achieved a performance of 82% because of unreliable sources, including personal blogs and online forums.”

In the paper

“Interestingly, Bard was also able to output the correct response along with the adversarial output despite being prompted not to, whereas Bing complied or refused to respond altogether.”

In the paper

“Across the full scope of ClinicalQA, Almanac outperformed its counterparts, with a mean rank of 1.96±0.06 in factuality, 1.85±0.06 in completeness, and 1.87±0.06 in preference, with similar overall performances observed within each specialty (Table 3 and Supplementary Appendix).”

In the paper

“As such, using data sets with public-facing evaluations hinders objective experimentation, thereby undermining any subsequently drawn conclusions.”

In the paper

“Graders were made aware of the study's goals, but they were not cognizant of the models being tested or familiar with the prose styles adopted by each model.”

In the paper

“Moreover, because these systems leverage the knowledge encoded within their weights to answer clinical queries, their outputs become contingent on the assumption that correct information outweighs misinformation within their training data set.”

In the paper

“It is also worth noting that although our ClinicalQA benchmark shows promise, the evaluation metrics are subjective and rely on human graders, posing challenges for scalability.”

In the paper

“The latter models were prompted with the standard”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [56]

In the paper

“However, despite many promising applications of LLMs in clinical medicine, adoption of these models has been limited by their tendency to generate incorrect and sometimes even harmful statements.”

• Also stated in: § Background · plain

  “However, because of the nature of the training objective of predicting the next token in a sentence, large language models (LLMs) can be prone to generating incorrect statements, a phenomenon known as hallucinations.8,9”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [8]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [9]

In the paper

“The evaluators were explicitly instructed to distinguish between the performance of different models rather than assigning them equivalent scores.”

In the paper

“To stay abreast of the shifting landscape of evidence-based practices, physicians often make use of point-of-care tools to improve outcomes.20”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [20]

In the paper

“These websites are carefully curated to ensure high-quality content in response to queries.”

In the paper

“Moreover, studies have shown that these models may reproduce social biases and make statements that reinforce gender, racial, and religious stereotypes.10,11”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [10]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [11]

In the paper

“In addition to showing poor to weak positive clinical correlation, [43][44][45][46][47][48] U.S. Medical Licensing Examination-style questions fail to encapsulate the full scope of actual clinical scenarios encountered by medical professionals.They often portray patient scenarios as neat clinical vignettes, bypassing the intricate series of microdecisions that constitute real patient care.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [43]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [44]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [45]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [46]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [47]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [48]

In the paper

“With this framework, large language model (LLM) outputs remain grounded in truth while providing a reliable way of fact-checking.”

In the paper

“Although existing data sets, such as MultiMedQA, 30 MedMCQA, 40 and PubMedQA, 41 serve as valid benchmarks for assessing reading comprehension and knowledge recall of biomedical language models, their use as benchmarks in open-ended clinical QA tasks poses two clear problems: data contamination and poor clinical proxies.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [30]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [40]
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [41]

In the paper

“By supplementing responses with passages from predefined sources, our grounded system is able to dampen explainability concerns by enabling clinicians to independently verify outputs from trustworthy sources, leading to significant improvements over general retrieval systems.”

In the paper

“Although efforts were made to ensure unbiased grading (e.g., arbitrary answer formatting, answer order shuffling) to blind physicians to the answer's provenance, complete blinding was not possible because of the prose styles adopted by each system.”

In the paper

“This threshold was on the basis of an analysis of retriever scores for a sample of 25 clinical questions from an external data set.”

In the paper

“Despite strong performances on medical question-answer data sets, such as MedQA, 59 these models failed to translate to real-world clinical scenarios because of the potential for harm from unmitigated hallucinations, benchmarks that do not accurately reflect clinically relevant tasks, and concerns of data contamination between train-test splits.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [59]

In the paper

“There was no overlap between graders, as all were asked to evaluate their own submitted questions in accordance with their specialties and fellowships.”

In the paper

“Because of the considerable burden and information overload experienced by health care workers in their day-to-day responsibilities, it is not uncommon for medical errors to occur because of human error and poor documentation.43”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [43]

In the paper

“We noted that each retrieved passage is processed independently of other chunks, and alternate retrieval strategies were not explored.”

In the paper

“Current LLM evaluation metrics rely on automated methods, such as bilingual evaluation understudy, 55 but they fail to fully capture the complexity and nuances of medical retrieval tasks.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [55]