Evaluating Metrics for Safety with LLM-as-Judges
Open accessAbstract
LLMs (Large Language Models) are increasingly used in text processing pipelines to intelligently respond to a variety of inputs and generation tasks. This raises the possibility of replacing human roles that bottleneck existing information flows, either due to insufficient staff or process complexity. However, LLMs make mistakes and some processing roles are safety critical. For example, triaging post-operative care to patients based on hospital referral letters, or updating site access schedules in nuclear facilities for work crews. If we want to introduce LLMs into critical information flows that were previously performed by humans, how can we make them safe and reliable? Rather than make performative claims about augmented generation frameworks or graph-based techniques, this paper argues that the safety argument should focus on the type of evidence we get from evaluation points in LLM processes, particularly in frameworks that employ LLM-as-Judges (LaJ) evaluators. This paper argues that although we cannot get deterministic evaluations from many natural language processing tasks, by adopting a basket of weighted metrics it may be possible to lower the risk of errors within an evaluation, use context sensitivity to define error severity and design confidence thresholds that trigger human review of critical LaJ judgments when concordance across evaluators is low.
AI snapshot
Written from the paper’s full text.
TL;DR
The paper argues that safety arguments for LLM deployments in critical information flows should focus on the evidence produced by evaluation points, especially LLM-as-Judge (LaJ) evaluators, rather than on performative claims about augmentation frameworks. Using an illustrative peri-operative risk assessment agentic framework, it shows that weighted, context-sensitive LaJ metrics can help structure assurance but that vague or non-deterministic scoring remains difficult to assure.
Key findings
- The paper proposes a basket of weighted metrics for LaJ evaluation: Coverage (30%), Critical Items (30%), Correctness & Specificity (20%), Prioritisation Alignment (10%), and Actionability Alignment (10%), aggregated to a 0–100 confidence score with grade bands High ≥90, Medium 70–89, Low 40–69, Very Low <40.
- Critical quality gates act as hard caps: a missed major gate caps the overall score at ≤69 and flags the case for human review; the critical-items subscore is capped at ≤0.40 for one major miss and ≤0.20 for multiple major misses.
- The illustrative Adjudicator LaJ compares an Anaesthetist agent’s brief against specialist agents’ briefs; disagreements are categorised as MISS, CONFLICT, OVERCALL, or AMBIGUOUS with minor/moderate/major severity.
- The paper finds that parts of the scoring rubric remain opaque: the LLM claims coverage = matched specialist items / total specialist items, but the prompt does not make this explicit; soft metrics like “correctness & specificity” lack quantified definitions, and capping only partially mitigates missed errors if both agents miss them.
Methods
- Design: conceptual/position paper with an illustrative agentic framework for peri-operative risk assessment; not a full system assurance or statistical evaluation.
- Data/sample: risk categories were generated from a small set of patient records (patient record set from HuggingFace), with an imaginary scenario involving Surgeon, Anaesthetist, specialist agents, and Adjudicator LaJ; developed with a clinical lead for critical care and anaesthesia.
- Analysis: qualitative examination of how LaJ evaluation points and weighted metrics could provide evidence for safety claims, including traceability to verbatim evidence extracts and dashboard scorecards; no statistical results on error detection or missed critical contexts because these are model dependent.
Limitations
- Stated: specialist profiles and quality gates were shortened to prevent agent misalignment/context poisoning; verbatim extracts were not easy to match with briefs, so human reviews could take time; no statistical results on how often errors were detected or critical contexts missed because these are model dependent.
- Stated: the implementation is an imaginary scenario and does not mirror clinical practice; the aim is not to assure a fully implemented system.
- Evident: the paper provides no empirical validation of LaJ reliability or of the proposed metric weights; it notes risks of hallucination, omission, and incompleteness by the LaJ itself remain, and that soft metrics dependent on LLM judgments remain difficult to assure.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
17 of 19 cited sources checked · 4 supported · 2 flagged · accuracy not yet measured
- Premisecitation§ 1.1 Assessing the performance of LLMs · p. 4
In a nuclear medicine Q&A system, string-based metrics (ROUGE / Levenshtein distance) did not correlate with radiologists' assessments of answer quality.
“For instance, in a nuclear medicine Q&A system, string-based metrics (ROUGE / Levenshtein distance) did not correlate with radiologists' assessments of answer quality [6].”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [6] 6. Y. Fukui, Y. Kawata, K. Kobashi, Y. Nagatani, and H. Iguchi. Evaluation of a retrieval-augmented generation system using a japanese institutional nuclear medicine manual and large language model-automated scoring. Radiological Physics and Technology , 18(3):861-876, 2025. source
In Results, Table 4 reports positive correlations between human answer-quality ratings and both ROUGE (r = 0.262–0.525) and Levenshtein distance (r = 0.332–0.489), rather than no correlation. Methods > Evaluation metrics also identifies the evaluators as two radiological technologists and a medical physicist, not radiologists. The Discussion does say these metrics sometimes diverged from human evaluations, but that does not establish the citing sentence's categorical claim. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
- Premisecitation§ 1.1 Assessing the performance of LLMs · p. 5
In nuclear medicine QA tasks, LLM-judged metrics only partially correlated with radiologists' assessments.
“For instance, in nuclear medicine QA tasks, LLM-judged metrics only partially correlated with radiologists' assessments, raising concerns about entrusting judgment to models that cannot fully grasp context-specific error severity [6].”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [6] 6. Y. Fukui, Y. Kawata, K. Kobashi, Y. Nagatani, and H. Iguchi. Evaluation of a retrieval-augmented generation system using a japanese institutional nuclear medicine manual and large language model-automated scoring. Radiological Physics and Technology , 18(3):861-876, 2025. source
The Results section, “Correlation between human and automated evaluations” (page unavailable), reports weak-to-moderate correlations and limited agreement between LLM-based scores and human ratings. But the Methods section, “Evaluation metrics” (page unavailable), identifies the human evaluators as two radiological technologists and a medical physicist, not radiologists. The paper also does not establish that the disagreement arose from models failing to grasp context-specific error severity. (LLM, conf=0.88)
Experimental — accuracy not yet measured
- Premisecitation§ 1.1 Assessing the performance of LLMs · p. 2
Due to context sensitivity, generic performance benchmarks carry no weight in most safety critical contexts.
“However, there is little agreement on how to evaluate these claims, and due to the context sensitivity problem, generic performance benchmarks carry no weight in most safety critical contexts (see references in [11]).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [11] 11. E. Lim, Y. V. He, J. Joselowitz, K. Preston, M. Chowdhury, and et al. MATRIX: Multi-Agent simulaTion fRamework for safe Interactions and conteXtual clinical conversational evaluation. arXiv: arxiv.org/abs/2508.19163 , 2025. source
The Introduction and Additional information sections say generic task-success and quality metrics miss clinically relevant hazards, and that safety evaluation needs context-specific, hazard-focused measures. They do not say generic benchmarks carry no weight, or establish that conclusion for most safety-critical contexts beyond clinical conversations. The cited paper does not establish the sentence’s broader claim about agreement on how to evaluate these claims. (LLM, conf=0.86)
Experimental — accuracy not yet measured
- Premisecitation§ 1.1 Assessing the performance of LLMs · p. 4
Automatic scores align poorly with human expert judgments.
“Research has found that such automatic scores align poorly with human expert judgments [6].”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [6] 6. Y. Fukui, Y. Kawata, K. Kobashi, Y. Nagatani, and H. Iguchi. Evaluation of a retrieval-augmented generation system using a japanese institutional nuclear medicine manual and large language model-automated scoring. Radiological Physics and Technology , 18(3):861-876, 2025. source
- Premisecitation§ 1.1 Assessing the performance of LLMs · p. 4
Standard NLP metrics e.g. BLEU, ROUGE often fail to capture the nuanced requirements of domains like medicine.
“As previously mentioned, standard NLP metrics e.g. BLEU, ROUGE (Recall-Oriented Understudy for Gisting Evaluation), often fail to capture the nuanced requirements of domains like medicine.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [6] 6. Y. Fukui, Y. Kawata, K. Kobashi, Y. Nagatani, and H. Iguchi. Evaluation of a retrieval-augmented generation system using a japanese institutional nuclear medicine manual and large language model-automated scoring. Radiological Physics and Technology , 18(3):861-876, 2025. source
The Discussion (page not available) says traditional string-based metrics such as ROUGE can undervalue semantically correct medical answers and sometimes show trends opposite to human evaluations. This supports the general point, but the paper does not assess or make that finding about BLEU, the other named example. (LLM, conf=0.84)
Experimental — accuracy not yet measured
- Premisecitation§ 1.1 Assessing the performance of LLMs · p. 5
Instability under prompt variations and adversarial phrasing undermines reproducibility of evaluations in regulated domains.
“The problem is compounded by instability under prompt variations and adversarial phrasing, which undermines reproducibility of evaluations in regulated domains [16].”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [16] 16. P. Ammanabrolu et al. Evaluating evaluation: A survey of evaluation methods for large language models. arxiv.org: arXiv:2402.00100 , 2024. source
The Introduction and Conclusions discuss diffuse axion backgrounds and their detection, not evaluation reproducibility. Searches for prompt variation, adversarial phrasing, and regulated domains found no passage addressing the claim. (LLM, conf=0.78)
Experimental — accuracy not yet measured
- Premisecitation§ 1.1 Assessing the performance of LLMs · p. 2
There is little agreement on how to evaluate claims that RAG, knowledge graphs, fine tuning, agentic frameworks, prompt and context engineering reduce model hallucination and errors.
“However, there is little agreement on how to evaluate these claims, and due to the context sensitivity problem, generic performance benchmarks carry no weight in most safety critical contexts (see references in [11]).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [11] 11. E. Lim, Y. V. He, J. Joselowitz, K. Preston, M. Chowdhury, and et al. MATRIX: Multi-Agent simulaTion fRamework for safe Interactions and conteXtual clinical conversational evaluation. arXiv: arxiv.org/abs/2508.19163 , 2025. source
The Introduction and Additional information (pages not provided) say that generic performance metrics miss clinically relevant, safety-critical failures and that evaluation needs to account for specific hazards. The paper does not establish a lack of agreement about how to evaluate whether RAG, knowledge graphs, fine-tuning, or prompt and context engineering reduce hallucinations and errors; nor does it make the broader claim that generic benchmarks carry no weight in most safety-critical contexts. (LLM, conf=0.82)
Experimental — accuracy not yet measured
- Premisecitation§ 1.1 Assessing the performance of LLMs · p. 5
Bias amplification is a recurring issue: evaluators inherit the training data biases of the underlying LLM, leading to uneven scoring across subdomains.
“Bias amplification is a recurring issue: evaluators inherit the training data biases of the underlying LLM, leading to uneven scoring across subdomains [21].”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [21] 21. T. Zhang, Y. Gu, J. Gonzalez, and M. I. Jordan. On the reliability of large language models as evaluators. arXiv preprint, arXiv:2403.12015 , 2024. source
The Experiments section (p. 6) describes CLIP-score evaluation of generated images, and Appendix A (p. 17) describes human evaluators and a precaution against bias. Neither addresses LLM-based evaluators inheriting training-data biases or uneven scoring across subdomains. Searches for those terms and for bias amplification found no supporting passage. (LLM, conf=0.78)
Experimental — accuracy not yet measured
- Premisecitation§ 1.1 Assessing the performance of LLMs · p. 6
Even with an extensive augmentation framework (RAGuard and SafetyClamp) around the LLM, the risks of omission in the LLM outputs remained despite being substantially reduced.
“For example, in [19] they found that even with an extensive augmentation framework (RAGuard and SafetyClamp) around the LLM, the risks of omission in the LLM outputs remained despite being substantially reduced.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [19] 19. C. Walker, K. Aslansefat, M. N. Akram, and Y. Papadopoulos. Raguard: A novel approach for in-context safe retrieval augmented generation for llms. arXiv: arxiv.org/abs/2509.03768 , 2025. source
Section 5.3 (p. 13) reports that RAGuard and SafetyClamp improved safety-clause retrieval, yet even the best configurations omitted at least one required clause in over 90% of queries. That supports a reduced but persistent risk of omission in retrieved context, not a measured finding about omissions in LLM outputs. The Conclusion (p. 15) identifies the quality and safety of generated responses as future work. (LLM, conf=0.88)
Experimental — accuracy not yet measured
- Premisecitation§ 4 Conclusions · p. 17
Despite attempting to mitigate risks by using evaluation points on the LLM outputs, risks of hallucination, omission and incompleteness by the LaJ itself remain, particularly in medical settings.
“Despite attempting to mitigate risks by using evaluation points on the LLM outputs, risks of hallucination, omission and incompleteness by the LaJ itself remain, particularly in medical settings [20].”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [20] 20. Z. Wu, K. Xie, B. Gu, N. Kr¨ uger, K. J. Lin, and J. Yang. Why chain of thought fails in clinical text understanding. arXiv preprint, arXiv:2arXiv.2509.21933 , 2025. source
Section 4.5 (p. 9) identifies hallucination, omission, and incompleteness in clinical LLM reasoning traces and describes expert checks of an LLM-as-a-Judge. But those are errors the judge evaluates, not failures attributed to the judge itself; the paper reports high agreement with expert annotations, while noting that expert validation was limited (Section 5, p. 10). (LLM, conf=0.84)
Experimental — accuracy not yet measured
- Interpretationcitation§ 1.1 Assessing the performance of LLMs · p. 6
Where omission is a substantial risk, the use of RAG or LaJ based augmented evaluation remains insufficient.
“Where omission is a substantial risk, the use of RAG or LaJ based augmented evaluation remains insufficient.”
Quote in the paperSubstantial claim - Premisecitation§ 1.1 Assessing the performance of LLMs · p. 5
Studies consistently show low reliability of LLM-based evaluators when domain expertise is required.
“While LLM-based evaluators have been tested as surrogates for expert panels in healthcare and safety assurance, studies consistently show low reliability when domain expertise is required [6][21], which means some mechanism is required to flag when human review is required to check the model's output.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [6][21]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning§ 3 Evaluating metrics for safety · p. 16
Risks that are not identified would not have the specificity metric applied, giving a false impression about the quality of the risk assessment comparison.
“Risks that are not identified would not have the metric applied (i.e. an undetected error), giving a false impression about the quality of the risk assessment comparison.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 3 Evaluating metrics for safety · p. 15
A common list of quality gates would not necessarily guarantee conformance over the input (the patient record).
“Perhaps this last issue brings us back to deploying a common list of quality gates agreed by clinical specialists so that agents deploy from a restrictive JSON schema under the STRICT output flag. This would guarantee a level of conformance to one vocabulary by the agents, but not necessarily over the input (i.e. the patient record).”
Quote in the paperSubstantial claim - Interpretationreasoning§ 3 Evaluating metrics for safety · p. 16
Increasing specificity might raise false positives for poor concordance causing unnecessary human review.
“Instead it might raise false positives for poor concordance causing unnecessary human review.”
Quote in the paperSubstantial claim - Interpretationreasoning§ 3 Evaluating metrics for safety · p. 16
Increasing the specificity does not lower risk for the patient in this context.
“Increasing the specificity does not lower risk for the patient in this context.”
Quote in the paperSubstantial claim- Rests on an assumption: The anaesthetist's assessment of low oxygen in this context is correct and knows what that means for the surgical procedure.
- May combine two claims. 25
- Classified as probably not a substantial claim. 59
May combine two claims.
Premisecitation§ 1.1 Assessing the performance of LLMs · p. 3Omission remains an outstanding problem for LLM evaluation, and to date the only means to check for omission is to go back to the source material and review it against the LLM output by hand.
“Omission remains an outstanding problem for LLM evaluation, and despite recent work by Microsoft [13] to date the only means to check for omission is to go back to the source material and review it against the LLM output by hand.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [13] 13. D. Metropolitansky and J. Larson. Towards effective extraction and evaluation of factual claims. In ACL 2025 Main Conference , February 2025. source
The Introduction (p. 1) recognizes that omitted factual content can compromise fact-checking, but it also says the paper introduces an automated method for assessing how well claims cover the source text. The Conclusion (p. 9) likewise describes automated, scalable coverage assessment. This contradicts the claim that checking for omissions can only be done by manual comparison. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 1.1 Assessing the performance of LLMs · p. 2Simply increasing input token lengths results in greater misalignment.
“given that simply increasing input token lengths results in greater misalignment [9]”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [9] 9. Levy, M. et al . The impact of input length on the reasoning performance of large language models. arxiv.org: 10.48550/arXiv.2402.14848 , 2024. source
Section 3.3 describes increasing input lengths while keeping the underlying task fixed. Section 7 reports that longer inputs led some models to disregard answer-format and reasoning-order instructions. That supports a narrower claim about instruction-following failures, but the paper does not establish greater misalignment in general. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Premisecitation§ 1.1 Assessing the performance of LLMs · p. 2Context is essential for judging LLM performance.
“As Chen et al. argue, context is essential for judging LLM performance [2]”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [2] 2. X. Chen, R. A. Chi, X. Wang, and D. Zhou. Premise order matters in reasoning with large language models. arxiv.org: 10.48550/arXiv.2402.08939 , 2024. source
The Abstract and Section 5 show that premise order substantially affects LLM performance on reasoning tasks, supporting the point that input context matters. They do not establish the broader claim that context is essential for judging LLM performance generally. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation§ 1.1 Assessing the performance of LLMs · p. 6Work on why chain-of-thought reasoning (including LLM-as-a-judge evaluation) fails in medical text understanding found that hallucination and information omission are dominant failure modes of CoT.
“What is imporant to note is that they found 'LLM-as-a-Judge framework with expert review also highlights the hallucination and information omission as dominant failure modes of CoT'”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [20] 20. Z. Wu, K. Xie, B. Gu, N. Kr¨ uger, K. J. Lin, and J. Yang. Why chain of thought fails in clinical text understanding. arXiv preprint, arXiv:2arXiv.2509.21933 , 2025. source
May combine two claims.
Premisecitation§ 1.1 Assessing the performance of LLMs · p. 4An LLM-based evaluation metric (RAGAS) achieved more consistent rankings with the experts, but still only modestly correlated to manual scores, implying that expert oversight remains essential.
“An LLM-based evaluation metric (RAGAS) achieved more consistent rankings with the experts, but still only modestly correlated to manual scores, implying that expert oversight remains essential.”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [6] 6. Y. Fukui, Y. Kawata, K. Kobashi, Y. Nagatani, and H. Iguchi. Evaluation of a retrieval-augmented generation system using a japanese institutional nuclear medicine manual and large language model-automated scoring. Radiological Physics and Technology , 18(3):861-876, 2025. source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation§ 1.1 Assessing the performance of LLMs · p. 6Numbers, units, and measurement-related tokens are overrepresented among CoT incorrect traces, as are abbreviations for laboratory and physiologic assessments.
“they 'observe that numbers, units, and measurement-related tokens are overrepresented among CoTincorrect traces (e.g., pt, ct, numeric values), as are abbreviations for laboratory and physiologic assessments such as plt (platelet count) and pft (pulmonary function test)'”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [20] 20. Z. Wu, K. Xie, B. Gu, N. Kr¨ uger, K. J. Lin, and J. Yang. Why chain of thought fails in clinical text understanding. arXiv preprint, arXiv:2arXiv.2509.21933 , 2025. source
Classified as probably not a substantial claim. May combine two claims.
Premisecitation§ 3 Evaluating metrics for safety · p. 15Medical abbreviations or chemical symbols are common in specialist domains but their usage varies.
“Medical abbreviations or chemical symbols are common in specialist domains but their usage varies [20].”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [20] 20. Z. Wu, K. Xie, B. Gu, N. Kr¨ uger, K. J. Lin, and J. Yang. Why chain of thought fails in clinical text understanding. arXiv preprint, arXiv:2arXiv.2509.21933 , 2025. source
Section 4.3 (p. 8) says medical concepts and numerical features frequently occur in clinical text and discusses misunderstandings of clinical abbreviations. The clinical example (pp. 23–24) illustrates an abbreviation being misread, but the paper does not establish that abbreviation usage varies or address chemical symbols in specialist domains. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Classified as probably not a substantial claim.
Findingown data§ 3 Evaluating metrics for safety · p. 15In the explanation of how the coverage figure is determined, the LLM claims coverage = (specialist items matched) / (total specialist items).
“Thus in the explanation of how the coverage fi gure is determined, the LLM claims: coverage = (specialist items matched) / (total specialist items) .”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone given§ 3 Evaluating metrics for safety · p. 15The anaesthetist's brief rewrote 'arterial oxygen saturation' as SpO2 instead of quoting verbatim from the patient record.
“For example, instead of quoting verbatim from the patient record, the anaesthetist's brief rewrote 'arterial oxygen saturation' as SpO 2 .”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Findingreasoning§ 3 Evaluating metrics for safety · p. 16The prompt for correctness & specificity does not define what 'penalise' means in terms of quantity, the range of contexts it could apply, or whether it is proportional to the contextual risk.
“There is no definition of what 'penalise' means in terms of quantity, the range of contexts it could apply and whether it is proportional to the contextual risk.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Findingnone given§ Additional Quality Gates (Cardiology) · p. 12Keeping the list of safety critical quality gates to minimal lengths greatly helped the consistency of the specialist agents' risk briefs.
“Keeping the list of safety critical quality gates to minimal lengths greatly helped the consistency of the specialist agents' risk briefs.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Findingreasoning§ 3 Evaluating metrics for safety · p. 16Closer examination proves that an LLM's claim of performing something that looks deterministic has little clarity.
“Although an LLM may claim it is performing something that looks deterministic, closer examination proves this has little clarity.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingreasoning§ 14 K. Clegg et al. · p. 14In practice, the verbatim extracts were not found easy to match with the briefs due to the complex text sometimes present in a patient's record.
“However, in practice the verbatim extracts were not found easy to match with the briefs due to the complex text sometimes present in a patient's record”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone given§ 3 Evaluating metrics for safety · p. 15SpO2 is one of several possible variants for this synonym in the patient records.
“This is one of several possible variants for this synonym in the patient records.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown data§ Additional Quality Gates (Cardiology) · p. 13In the authors' implementation, disagreements between the anaesthetist's brief and specialist briefs tended to be mostly misses.
“Disagreements in our implementation tended to be mostly misses”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Findingown data§ 3 Evaluating metrics for safety · p. 15The LLM-derived prompt does not make the coverage formula explicit; it states COVERAGE (0.30): % of Speciality's salient risks present in BRIEF S that BRIEF A also captures (allow synonymous phrasing).
“But when we examine the LLM derived prompt, it does not make this explicit: COVERAGE (0.30): % of Speciality's salient risks present in BRIEF S that BRIEF A also captures (allow synonymous phrasing) .”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable).
Findingreasoning§ 3 Evaluating metrics for safety · p. 16No evidence can be obtained about what 'factually supported' means in terms of the metric.
“Likewise, no evidence can be obtained about what 'factually supported' means in terms of the metric.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationreasoning§ 1.1 Assessing the performance of LLMs · p. 2When evaluating LLM performance against what good looks like, we need to do more than simply compare LLM outputs to human performance; we need to ensure we can also detect and handle errors that are unique to LLMs.
“But when we try to evaluate LLM performance against what good looks like, we need to do more than simply compare LLM outputs to human performance, we need to ensure we can also detect and handle errors that are unique to LLMs.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone given§ 1.1 Assessing the performance of LLMs · p. 6Most LaJ use cases that report adequate performance in safety critical contexts tend to be classification type tasks, and as such do not involve risks of omission.
“It is worth mentioning that most of the LaJ use cases that report adequate performance in safety critical contexts tend to be classification type tasks, and as such do not involve risks of omission.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationcitation§ 1.1 Assessing the performance of LLMs · p. 5Some mechanism is required to flag when human review is required to check the model's output.
“While LLM-based evaluators have been tested as surrogates for expert panels in healthcare and safety assurance, studies consistently show low reliability when domain expertise is required [6][21], which means some mechanism is required to flag when human review is required to check the model's output.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [6][21]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisereasoning§ 1.1 Assessing the performance of LLMs · p. 2Small changes in task complexity can have a disproportionately adverse effect on output quality.
“For example, sparse exposure to training data (typical of restricted private domains) can mean LLM outputs show higher variance on the same task and small changes in task complexity can have a disproportionately adverse effect on output quality.”
Quote in the paperNot a substantial claim- Rests on an assumption: 'Complexity' across different text processing tasks has no recognised metric.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Interpretationreasoning§ 1.1 Assessing the performance of LLMs · p. 6This implies that a method for detecting the omission in LLM output can be assured.
“This implies that a method for detecting the omission in LLM output can be assured.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationnone given§ 1.1 Assessing the performance of LLMs · p. 2Many architectural proposals can be affected by or hide unexpected complexity in their evaluations.
“However, many of the architectural proposals can be affected by or hide unexpected complexity in their evaluations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationreasoning§ 1.1 Assessing the performance of LLMs · p. 2The burden of proof moves to the robustness of the evaluation points in the process, regardless of whether that evaluation is by machine or human.
“The 'burden of proof' moves to the robustness of the evaluation points in the process, regardless of whether that evaluation is by machine or human.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationreasoning§ 1.1 Assessing the performance of LLMs · p. 1Context dependence can mean that not all errors in the output are of equal severity even when they are of the same type; where the error occurs may be as critical as the error type.
“Context dependence can mean that not all errors in the output are of equal severity even when they are of the same type, i.e. where the error occurs (its context) may be as critical as the error type.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Interpretationreasoning§ 1.1 Assessing the performance of LLMs · p. 2The method chosen to achieve the quality of performance needed is irrelevant, provided that evidence from evaluations demonstrates that the LLM is safe to use for the specific task under consideration in its particular usage context.
“To some extent the method chosen to achieve the quality of performance needed is irrelevant, provided that evidence from evaluations demonstrates that the LLM is safe to use for the specific task under consideration in its particular usage context.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Premisenone given§ 1.1 Assessing the performance of LLMs · p. 2For critical tasks, there may be limited or even no prior data on the types of errors that can be made, or their severity, as critical tasks tend to be performed by suitably qualified or experienced personnel (SQEP) who ensure high quality standards.
“For critical tasks, there may be limited or even no prior data on the types of errors that can be made, or their severity, as critical tasks tend to be performed by suitably qualified or experienced personnel (SQEP) who ensure high quality standards.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Interpretationnone given§ 1.1 Assessing the performance of LLMs · p. 6These are exactly the types of text items we are trying to assure in our experiment.
“As we show later, these are exactly the types of text items we are trying to assure in our experiment (see Section 2.2).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationreasoning§ 1.1 Assessing the performance of LLMs · p. 6The assurance argument in these cases rests on whether exposure to the risk of omission is low, or if the exposure is high, then the safety case should argue that an omission will not contribute to known hazards.
“The assurance argument in these cases rests on whether exposure to the risk of omission is low, or if the exposure is high (i.e. it could occur in many contexts), then the safety case should argue that an omission will not contribute to known hazards.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone given§ 1.1 Assessing the performance of LLMs · p. 2Sparse exposure to training data (typical of restricted private domains) can mean LLM outputs show higher variance on the same task.
“For example, sparse exposure to training data (typical of restricted private domains) can mean LLM outputs show higher variance on the same task and small changes in task complexity can have a disproportionately adverse effect on output quality.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationreasoning§ 1.1 Assessing the performance of LLMs · p. 2The context sensitivity problem means that detecting errors unique to LLMs might require greater contextual understanding than the model can be provided with as part of its inputs or prompt, therefore additional safeguards are necessary.
“The context sensitivity problem means that detecting these sorts of errors might require greater contextual understanding than the model can be provided with as part of its inputs or prompt for the task, therefore additional safeguards are necessary.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone given§ 1.1 Assessing the performance of LLMs · p. 6Whether such arguments are deemed sufficient will depend on what type of evidence is presented to back up those claims and who is making the safety assessment.
“Whether such arguments are deemed sufficient will depend on what type of evidence is presented to back up those claims and who is making the safety assessment.”
Quote in the paperNot a substantial claimMay combine two claims.
Premisecitation§ 1.1 Assessing the performance of LLMs · p. 5LaJs may be prone to issues of bias, reproducibility, and variance in their judgments, and there is the issue of 'who judges the judge?'.
“In addition to what has already been discussed, LaJs may be prone to issues of bias, reproducibility, and variance in their judgments, and there is the issue of 'who judges the judge?'.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable). May combine two claims.
Interpretationreasoning§ 1.1 Assessing the performance of LLMs · p. 6LaJ-based systems require a rigorous evaluation of the LLM as judge to make the safety case sufficient, and these may rely on properties intrinsic to the model.
“While this study claims the best LLMs 'out performed clinicians in a blinded assessment of 240 dialogues', LaJ-based systems such as this require a rigorous evaluation of the LLM as judge to make the safety case sufficient, and these may rely on properties intrinsic to the model.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationnone given§ 2.1 Patient safety under anaesthetic · p. 9The imaginary scenario, while not mirroring clinical practice in hospitals, serves as a realistic context for assessing LaJ evaluations in terms of their safety implications.
“Although this does not mirror clinical practice in hospitals, it serves as a realistic context in which we can assess LaJ evaluations in terms of their safety implications.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoning§ 2.2 Implementation · p. 10The risk taxonomy provided a consistent way to capture and compare risks across different patients and specialist contexts, ensuring that categories were consistent and risks could be linked to categorised domains rather than left as free text extractions.
“This taxonomy provided a consistent way to capture and compare risks across different patients and specialist contexts, ensuring that the categories were consistent and risks could be linked to categorised domains rather than left as free text extractions.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone given§ 2 LaJ Use Case: An agentic framework for Peri-operative Risk Assessment · p. 7The medical domain has error severity dependent on its context.
“To illustrate the problem of context sensitivity described earlier, we chose the medical domain where the error severity is dependent on its context.”
Quote in the paperNot a substantial claim- Rests on an assumption: The medical domain is a good fit for an agentic LLM workflow.
Probably not a substantial claim: may not pass S4 (based).
Interpretationreasoning§ 2.2 Implementation · p. 11The adjudicator's output of a machine-readable scorecard and a short explanatory note in theory gives a human and machine auditable measure of how well the Anaesthetist agent's risk assessment aligns with specialist agents' expertise.
“The adjudicator outputs both a machine-readable scorecard (JSON: sub scores, agreements, disagreements, gate checks) and a short explanatory note, in theory giving a human and machine auditable measure of how well the Anaesthetist agent's risk assessment aligns with specialist agents' expertise.”
Quote in the paperNot a substantial claim- Rests on an assumption: The specialist agents' risk assessments are accurate enough to serve as a reference standard for evaluating the anaesthetist agent's risk assessment.
Probably not a substantial claim: may not pass S4 (based).
Interpretationnone given§ 1.1 Assessing the performance of LLMs · p. 6Properties intrinsic to the model can be difficult to verify, especially for large models and limited amounts of testing.
“Such properties can be difficult to verify, especially for large models and limited amounts of testing.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoning§ 14 K. Clegg et al. · p. 14Every claim in the briefs maps back to verbatim extracted JSONL lines from the patient record (source id + locator + extract text).
“therefore every claim in the briefs maps back to verbatim extracted JSONL lines from the patient record (source id + locator + extract text).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable). May combine two claims.
Interpretationreasoning§ 4 Conclusions · p. 17Aspects of text-based comparisons, such as gaining assurance that synonyms will be recognised equally, or the use of unquantified criteria like 'factually supported' that are dependent on LLM judgments, will remain difficult to assure.
“Unfortunately there are aspects of text based comparisons, for example gaining assurance that synonyms will be recognised equally, or the use of unquantified criteria like 'factually supported' that are dependent on LLM judgments and which will remain difficult to assure.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationreasoning§ 3 Evaluating metrics for safety · p. 15There is a risk that if the LLM is asked to create the metric, it will propose it but not declare it explicitly in the prompt.
“There is a risk if the LLM is asked to create the metric, it will propose it but not declare it explicitly in the prompt.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable). May combine two claims.
Interpretationreasoning§ 1.1 Assessing the performance of LLMs · p. 5While RAG and KGs can mitigate some weaknesses of stand-alone LLMs, the challenge of rigorously evaluating and assuring their performance in safety-critical contexts remains a significant problem.
“While RAG and KGs can mitigate some weaknesses of stand-alone LLMs, the challenge of rigorously evaluating and assuring their performance in safetycritical contexts remains a significant problem.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone given§ 2.1 Patient safety under anaesthetic · p. 7It is possible for the clinic to miss some aspect of the patient's condition or underestimate a complex risk scenario.
“However, it is possible for the clinic to miss some aspect of the patient's condition or underestimate a complex risk scenario.”
Quote in the paperNot a substantial claim- Rests on an assumption: The illustrative peri-operative risk assessment agentic framework is representative of real-world safety-critical LaJ use cases.
- Rests on an assumption: The bleomycin exposure scenario is a realistic and relevant example of a context-sensitive safety-critical error.
May combine two claims.
Interpretationreasoning§ 3 Evaluating metrics for safety · p. 16The safeguard of capping a concordance score if a critical risk has been missed by the anaesthetist agent only partially mitigates the false impression, because if both miss the error it would be undetected.
“The safeguard of capping a concordance score if a critical risk has been missed by the anaesthetist agent only partially mitigates this, if both miss the error would be undetected.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductreasoning§ 2.2 Implementation · p. 11In the implementation, shortening most of the specialist profiles prevented agents from misaligning during their tasks.
“It should be noted that in the implementation we had to shorten most of the specialist profiles to prevent agents misaligning during their tasks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Premisenone given§ 2.1 Patient safety under anaesthetic · p. 8Critical information like bleomycin history can be difficult to find in the patient's record.
“Critical information like this can be difficult to find in the patient's record.”
Quote in the paperNot a substantial claim- Rests on an assumption: The bleomycin exposure scenario is a realistic and relevant example of a context-sensitive safety-critical error.
Classified as probably not a substantial claim.
Interpretationreasoning§ 4 Conclusions · p. 17Asking an LLM how it calculates coverage is insufficient evidence to use in a safety claim.
“We can ask an LLM how it calculates coverage, but that is insufficient evidence to use in a safety claim.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone given§ 2.2 Implementation · p. 10Each specialist agent's defined profile and prompt tailored to its domain ensure its output is a focused summary of the most critical risks for that speciality with supporting evidence extracted verbatim from the patient's record.
“Each specialist agent (e.g. Cardiologist , Respiratory , Nephrologist , Haematology , etc) has a defined profile and prompt tailored to its domain, ensuring that its output is a focused summary of the most critical risks for that speciality together with supporting evidence extracted verbatim from the patient's record.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationreasoning§ 4 Conclusions · p. 17Parts of the scoring rubric remain shrouded in mystery.
“However, parts of the scoring rubric remain shrouded in mystery.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.
Interpretationreasoning§ 2.1 Patient safety under anaesthetic · p. 8If bleomycin exposure is missed by the anaesthetist, the most likely prevention would be at the earlier pre-op clinic (by nurses or consultant anaesthetist reviewing the record) or by the oncologist's treatment summary being reviewed pre-surgery.
“If bleomycin exposure is missed by the anaesthetist, the most likely prevention would be at the earlier pre-op clinic (by nurses or consultant anaesthetist reviewing the record) or by the oncologist's treatment summary being reviewed pre-surgery.”
Quote in the paperNot a substantial claim- Rests on an assumption: The bleomycin exposure scenario is a realistic and relevant example of a context-sensitive safety-critical error.
Probably not a substantial claim: may not pass S4 (based).
Interpretationreasoning§ Additional Quality Gates (Cardiology) · p. 13In theory, the hard caps prevent superficial agreement from obscuring dangerous omissions.
“In theory this prevents superficial agreement from obscuring dangerous omissions (however, see discussion in Section 3).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning§ 14 K. Clegg et al. · p. 14Human reviews could take time to confirm the assessment was correct.
“meaning that human reviews could take time to confirm the assessment was correct.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationreasoning§ 3 Evaluating metrics for safety · p. 16The LaJ is generating a numerical score based on vaguely defined judgements that directly affect the concordance grade, which is intended to be a safety assured metric.
“The LaJ is generating a numerical score based on vaguely defined judgements that directly affect the concordance grade, which is intended to be a safety assured metric.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoning§ 14 K. Clegg et al. · p. 14Traceability of assessment judgments can be accessed through verbatim evidence links and structured sub scores.
“Traceability of assessment judgments can be accessed through verbatim evidence links and structured sub scores”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoning§ Additional Quality Gates (Cardiology) · p. 13Critical gates act as hard caps: if a major gate is missed, the overall score is capped at ≤ 69 and the case is flagged for human review.
“Critical gates act as hard caps: if a major gate is missed (e.g. 'Acute LRTI ⇒ delay elective'), the overall score is capped at ≤ 69 and the case is flagged for human review.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable). May combine two claims.
Interpretationreasoning§ 4 Conclusions · p. 17An LLM can produce a chain-of-thought explanation of how it arrives at a score, but that does not guarantee what it is doing 'under the hood'.
“An LLM can produce a chain-of-thought explanation of how it arrives at a score, but that does not guarantee what it is doing 'under the hood'.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning§ 2.2 Implementation · p. 11The misalignment observed when specialist profiles were lengthy is a case of 'context poisoning', where a lengthy contextual input starts to confuse an agent.
“This seems to be a case of 'context poisoning', i.e. where a lengthy contextual input starts to confuse an agent.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationnone given§ 4 Conclusions · p. 17Traceable evidence of a calculation can only be obtained if the calculation is passed to an external tool or program code whose calculations can be traced.
“We can only get that type of evidence if the calculation is passed to an external tool or program code whose calculations we can trace.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationreasoning§ 3 Evaluating metrics for safety · p. 15Deploying a common list of quality gates agreed by clinical specialists so that agents deploy from a restrictive JSON schema under the STRICT output flag would guarantee a level of conformance to one vocabulary by the agents.
“Perhaps this last issue brings us back to deploying a common list of quality gates agreed by clinical specialists so that agents deploy from a restrictive JSON schema under the STRICT output flag. This would guarantee a level of conformance to one vocabulary by the agents, but not necessarily over the input (i.e. the patient record).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone given§ 4 Conclusions · p. 16The metric seems well designed for a safety critical context: it is risk-based, context sensitive and can be adjusted to align with local clinical policies or patient intake.
“The metric seems well designed for a safety critical context: it is risk-based, context sensitive and can be adjusted to align with local clinical policies or patient intake.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone given§ 2.2 Implementation · p. 9Error detection rates and missed critical contexts in agent outputs are model dependent.
“We do not present statistical results of how often errors were detected in an agent's output or how often the anaesthetist's brief was found to have missed critical contexts, as these are model dependent.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationreasoning§ 3 Evaluating metrics for safety · p. 16Perfect specificity only means that for those risks identified, there was close agreement.
“But perfect specificity only means for those risks identified, there was close agreement.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Premisenone given§ 2 LaJ Use Case: An agentic framework for Peri-operative Risk Assessment · p. 7The patient's condition and treatment can provide multiple overlapping contexts for risk assessment and much of the information that is stored is text based.
“The patient's condition and treatment can provide multiple overlapping contexts for risk assessment and much of the information that is stored is text based.”
Quote in the paperNot a substantial claim- Rests on an assumption: The medical domain is a good fit for an agentic LLM workflow.
The paper relies on this without support.
'Complexity' across different text processing tasks has no recognised metric.
“But 'complexity' across different text processing tasks has no recognised metric 3”
The paper relies on this without support.
The illustrative peri-operative risk assessment agentic framework is representative of real-world safety-critical LaJ use cases.
“For these reasons we decided to implement a simple agentic framework, with help from a clinical lead for critical care and anaesthesia, for peri-operative risk assessment clinic that could demonstrate a realistic LaJ task in a safety critical context.”
The paper relies on this without support.
The medical domain is a good fit for an agentic LLM workflow.
“This type of team work, comprising of domain specialists and a coordinator is a good fit for an agentic LLM workflow.”
The paper relies on this without support.
The bleomycin exposure scenario is a realistic and relevant example of a context-sensitive safety-critical error.
“An example of this is a patient who has, at some point in the past, received bleomycin (a chemotherapy agent).”
The paper relies on this without support.
If a patient risk assessment misses a condition like bleomycin exposure, serious harm to the patient could occur.
“given that if the risk assessment were to miss a condition like bleomycin exposure, serious harm to the patient could occur.”
The paper relies on this without support.
The specialist agents' risk assessments are accurate enough to serve as a reference standard for evaluating the anaesthetist agent's risk assessment.
“The Adjudicator agent (i.e. LaJ) then compares the Anaesthetist agent's risk assessment with that from each of the specialist agents.”
The paper relies on this without support.
The predefined quality gates are valid critical domain-specific safety rules.
“with special attention to predefined quality gates (critical domain-specific safety rules).”
The paper relies on this without support.
A real-world implementation of the described system would be classified as a medical device and require a full safety audit.
“If this were a real-world implementation, it would be classified as a medical device and have to undergo a full safety audit.”
The paper relies on this without support.
The anaesthetist's assessment of low oxygen in this context is correct and knows what that means for the surgical procedure.
“The anaesthetist's assessment of low oxygen in this context is correct and knows what that means for the surgical procedure.”
The paper relies on this without support.
The example metric is of the type typically used to argue that a LaJ-based evaluation can be trusted.
“We have presented an example of a composable, weighted metric of the type typically used to argue that a LaJ-based evaluation can be trusted.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Coverage is called a Jaccard index but defined as a different ratio
MinorThe stated formula divides matched items by all specialist items. A Jaccard index divides matched items by all distinct items in both briefs, so the two measures differ whenever the anaesthetist brief contains additional risk items. The 8-of-10 example agrees with both only if there are no additional items.
“1. Coverage (30%) - proportion of specialist risk items also present in the anaesthetist brief. This is a Jacquard Index overlap calculation. Example: 8 of 10 items matched ⇒ coverage = 0.8.”
“coverage = (specialist items matched) / (total specialist items) .”
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 1× · see citing works
“The first was a simple one-to-one token match, the second used BLEU (Bilingual Evaluation Understudy) score [17], which is a metric used to evaluate the quality of machine-translated text by comparing it to human-translated reference text, and thirdly cosine similarity, which is a measure used to…”
Hicks, M. T., Humphries, J., & Slater, J. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09775-5 · Cited by 456 · cited 1× · see citing works
“The report cites a heavily publicised paper from the University of Glasgow that was the first to describe LLMs as “bullshitters [sic]” [8], based on the philosopher Harry Frankfurt’s definition of someone who has “no concern for truth or how things really are” [5].”
Levy, M., Jacoby, A., & Goldberg, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.14848 · Cited by 119 · see citing works
Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review
Neha, F., Bhati, D., & Shukla, D. K. (2025) · AI · doi:10.3390/ai6090226 · Cited by 117 · cited 9× · see citing works
“In medical decision support, graph-augmented RAG pipelines have shown gains in precision and factual accuracy by linking patient queries to trusted knowledge bases of clinical relationships [14].”
InfluentialFast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation
Sauer, A., Boesel, F., Dockhorn, T., Blattmann, A., Esser, P., & Rombach, R. (2024) · arXiv · doi:10.48550/arxiv.2403.12015 · Cited by 76 · see citing works
Retrieval-augmented generation elevates local LLM quality in radiology contrast media consultation
Wada, A., Tanaka, Y., Nishizawa, M., Yamamoto, A., Akashi, T., Hagiwara, A., Hayakawa, Y. K., Kikuta, J., Shimoji, K., Sano, K., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01802-z · Cited by 58 · cited 3× · see citing works
“One recent radiology study showed that even with RAG improvements, a local 11B parameter model still lagged behind larger cloud LLMs on overall performance metrics [18], underlining the difficulty of closing the gap in critical tasks.”
InfluentialA comprehensive taxonomy of hallucinations in Large Language ModelsPreprint
Cossio, M. (2025) · arXiv.org · doi:10.48550/arxiv.2508.01781 · Cited by 30 · see citing works
Fukui, Y., Kawata, Y., Kobashi, K., Nagatani, Y., & Iguchi, H. (2025) · Radiological Physics and Technology · doi:10.1007/s12194-025-00941-y · Cited by 9 · cited 4× · see citing works
“…LLM-based evaluators have been tested as surrogates for expert panels in healthcare and safety assurance, studies consistently show low reliability when domain expertise is required [6][21], which means some mechanism is required to flag when human review is required to check the model’s output.”
InfluentialAutomatic Instantiation of Assurance Cases from Patterns Using Large Language ModelsPreprint
Odu, O., Belle, A. B., Wang, S., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2410.05488 · Cited by 9 · cited 2× · see citing works
“For example they cite the example of [15] who asked the same query of an LLM five times and took an average of the answers using a BLEU score in order to deal with output variance.”
Towards Effective Extraction and Evaluation of Factual Claims
Metropolitansky, D., & Larson, J. (2025) · arXiv · doi:10.48550/arxiv.2502.10855 · Cited by 7 · cited 1×
“Omission remains an outstanding problem for LLM evaluation, and despite recent work by Microsoft [13] to date the only means to check for omission is to go back to the source material and review it against the LLM output by hand.”
Towards Effective Extraction and Evaluation of Factual Claims
Metropolitansky, D., & Larson, J. (2025) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/2025.acl-long.348 · Cited by 7
Using GPT-4 to Generate Failure Logic
Clegg, K. D., Habli, I., & McDermid, J. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-68738-9_11 · Cited by 6 · see citing works
Self-citationRAGuard: A Novel Approach for In-Context Safe Retrieval Augmented Generation for LLMs
Walker, C., Aslansefat, K., Akram, M. N., & Papadopoulos, Y. I. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05073-1_13 · Cited by 4 · cited 1× · see citing works
“For example, in [19] they found that even with an extensive augmentation framework (RAGuard and SafetyClamp) around the LLM, the risks of omission in the LLM outputs remained despite being substantially reduced.”
Eby, J., & Takhistov, V. (2024) · arXiv · doi:10.48550/arxiv.2402.00100 · Cited by 4 · see citing works
Premise Order Matters in Reasoning with Large Language Models
Chen, X., Chi, R. A., Wang, X., & Zhou, D. (2024) · arXiv · doi:10.48550/arxiv.2402.08939 · Cited by 3 · see citing works
Why Chain of Thought Fails in Clinical Text Understanding
Wu, J., Xie, K., Gu, B., Krüger, N., Lin, K. J., & Yang, J. (2025) · arXiv · doi:10.48550/arxiv.2509.21933 · cited 3×
“Medical abbreviations or chemical symbols are common in specialist domains but their usage varies [20].”
InfluentialLim, E., He, Y. V., Joselowitz, J., Preston, K., Chowdhury, M., Williams, L., Higham, A., Mason, K., Melo, M., Lawton, T., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.19163 · cited 2×
“However, there is little agreement on how to evaluate these claims, and due to the context sensitivity problem, generic performance benchmarks carry no weight in most safety critical contexts (see references in [11]).”
Self-citationRAGuard: A Novel Approach for in-context Safe Retrieval Augmented Generation for LLMs
Walker, C., Aslansefat, K., Akram, M. N., & Papadopoulos, Y. (2025) · arXiv · doi:10.48550/arxiv.2509.03768
Anaesthetic implications of chemotherapy
Allan, N., Siller, C., & Breen, A. (2012) · Continuing Education in Anaesthesia Critical Care & Pain · doi:10.1093/bjaceaccp/mkr055
Flagged first, then by citations
2 of the 22 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- [12]12. Y. Liu, J. Wei, P.-Y. Chen, et al. Trustworthy ai: A survey and future directions. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI) , pages 7465-7472, 2023.
- Authors
- Clegg, K., Hawkins, R., Habli, I., & Lawton, T.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2512.15617
- OpenAlex
- W7116052653
- Semantic Scholar
- f04ed582f5653e1b2e8c8cb759d3519fa5b1a146
- Cited by
- 0
- References
- 22
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- LLM-as-a-judge, safety assessment, error severity, confidence threshold, human review, safety-critical systems, LLM reliability, context sensitivity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
