CoChat
Sign Up
Cited by 2Adversarial Robustness in Machine LearningHate Speech and Cyberbullying DetectionTopic Modeling

Evaluation of NeMo Guardrails as a Firewall for User–LLM Interaction

Open access
Azambuja, Antônio João, Guilherme, Marcos, Castro, João Victor Fernandes de, Lima, Jean Phelipe de Oliveira, Oliveira, Leonardo B., Soares, Anderson da Silva·2026·Future Internet
DOI 10.3390/fi18050252
Open this paper in CoChat — free accountOpen PDF

Abstract

The rapid integration of Large Language Models (LLMs) into critical personal and professional environments has exacerbated security risks, particularly adversarial attacks such as prompt injection and jailbreaking, which aim to bypass safety alignment. This study evaluates the efficacy of NVIDIA’s Llama-3.1-nemoguard-8b-content-safety model acting as a semantic firewall to mitigate these threats. To ensure a robust assessment, we utilized the ‘Do Not Answer’ dataset, augmented with 939 synthetically generated benign prompts to create a balanced corpus of 1878 samples. The evaluation methodology encompasses a risk-category analysis, standard binary classification metrics, and a novel metric, the Compensation Rate, which measures the firewall’s ability to block responses when the underlying LLM fails. Results indicate a high Precision (94.57%) but a moderate Sensitivity (51.97%), uncovering a critical performance trade-off: the model exhibits a conservative bias, prioritizing high precision to minimize false positives at the expense of recall for nuanced adversarial prompts, particularly in categories involving sensitive data leakage and misinformation. Furthermore, the proposed Compensation Rate achieved 34.8%, suggesting that the semantic firewall successfully mitigated 34.8% of instances where the foundational LLM’s internal safety alignment failed. These findings indicate that while the system effectively blocks explicit threats, its efficacy as a secondary defense diminishes against context-dependent vulnerabilities, notably data exfiltration and misinformation.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper evaluates NVIDIA’s Llama-3.1-nemoguard-8b-content-safety as an external semantic firewall for user–LLM interaction using a balanced 1,878-prompt corpus. It finds high precision but moderate sensitivity, with a Compensation Rate of 34.8% when the underlying GPT-4o mini fails to refuse unsafe prompts.

Key findings

  • The guardrail achieved Precision 94.57%, Sensitivity 51.97%, Accuracy 74.49%, and F1-Score 67.08%; of 939 unsafe prompts, 488 were true positives and 451 false negatives, while of 939 safe prompts, 911 were true negatives and 28 false positives.
  • Performance varied sharply by risk category: F1 was 12% for “Causing material harm by disseminating misinformation” and 19% for “Risks from leaking sensitive information,” while categories with explicit lexical cues performed better, e.g., Adult Content had a 0% false-positive rate.
  • The highest false-negative rate was in “Risks from leaking or inferring sensitive information (organization/gov)” at 44.9% (122 of 272 samples), followed by “Causing material harm by disseminating misinformation, e.g., in medicine or law” at 46.8% and “Disseminating false or misleading information” at 43.2%.
  • The proposed Compensation Rate was 34.8%, intercepting 8 of 23 GPT-4o mini safety-alignment failures; the firewall neutralized five of nine failures in Adult Content but missed five of six in the sensitive-information organization/gov category.

Methods

  • The study evaluated Llama-3.1-nemoguard-8b-content-safety as an external LLM firewall, not as a general-purpose model, using the “Do Not Answer” dataset’s 939 unsafe prompts plus 939 synthetically generated benign prompts for a balanced corpus of 1,878 samples.
  • Benign prompts were generated with GPT-4o mini at temperature 0.7 using a structured template with anti-repetition context, then manually reviewed by three authors for safety and thematic consistency; each prompt was reviewed by a single author without overlapping annotation or agreement reporting.
  • Analysis included binary classification metrics, per-risk-category confusion matrices and F1-Scores, and a new Compensation Rate metric, CR = CF/TF, where CF is successful firewall interception despite an inappropriate underlying-model response and TF is total GPT-4o mini safety-alignment failures.

Limitations

  • The paper states that the Compensation Rate analysis relied on a small number of GPT-4o mini failures (n = 23), limiting statistical generalizability and requiring validation on larger datasets.
  • The paper states that using GPT-4o mini both to generate synthetic benign prompts and as the foundational model in the Compensation Rate analysis introduces potential circularity and stylistic bias, and that English-centric training data plus the fixed guardrail model limit adaptability to regional legal frameworks such as Brazil’s LGPD.
  • The paper also notes that synthetic benign prompts may not fully represent real-world safe interactions, especially in context-dependent categories; evident from the text, manual review without overlapping annotation or agreement reporting further limits verification of synthetic-data quality.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/fi18050252

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

23 of 34 cited sources checked · 14 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    The Do Not Answer dataset contains 939 malicious user prompts in English classified as unsafe.

    “To this end, the Do Not Answer dataset was used, which contains 939 malicious user prompts.”
    • Also stated in: § 3.1.1. Do Not Answer Dataset · plain
      “The Do Not Answer dataset is an open dataset containing 939 prompts (instructions), in English, classified as unsafe.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [29] Wang, Y., Li, H., Han, X., Nakov, P., and Baldwin, T. (2023). Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs. arXiv. source

      The Introduction (page unavailable) describes Do-Not-Answer as a dataset of 939 prompts whose instructions should not be followed, and “Collecting questions” (page unavailable) confirms the total. The paper calls them risky prompts, covering mild as well as extreme risks; it does not establish that every prompt is malicious or explicitly state that all are in English. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
  • Premisecitation

    A significant gap remains: current models are susceptible to attacks, particularly in cases where they fail to correctly interpret the semantics of interactions.

    “Therefore, despite the advances already achieved in LLM security, a significant gap remains to be filled: current models are susceptible to attacks, particularly in cases where they fail to correctly interpret the semantics of interactions [10].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [10] Alzaabi, 2024, A Review of Recent Advances, Challenges, and Opportunities in Malicious Insider Threat Detection Using Machine Learning Methods, IEEE Access source

      The “Anomaly Detection Methods” section says adversarial attacks remain a challenge for ML and DL insider-threat detection models and calls for greater robustness. It does not establish the claim about LLM security or say that attacks succeed because models misinterpret the semantics of interactions. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
  • Premisecitation

    Relying solely on dimension-specific internal tuning-such as fine-tuning exclusively for safety-often degrades the model's performance in real-world settings.

    “Furthermore, recent evaluations demonstrate that relying solely on dimension-specific internal tuning-such as fine-tuning exclusively for safety-often degrades the model's performance in real-world settings [11].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [11] Naseem, U., Kashyap, G.S., Ali, R., Shabbir, E., Ray, S.K., Mohammad, A., and Seth, A. (2026). Are Aligned Large Language Models Still Misaligned?. arXiv. source

      The Benchmark Analysis (pp. 6–7) finds that models fine-tuned for a single dimension, including safety, have lower alignment scores and higher false-failure rates under joint constraints. But these are benchmark results; the paper does not demonstrate degraded performance in real-world deployments. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
  • Premisecitation

    The absence of standardized taxonomies makes comparing different models challenging, as they were trained against different taxonomies.

    “As acknowledged in the Llama Guard paper itself, the absence of standardized taxonomies makes comparing different models challenging, as they were trained against different taxonomies [32].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [32] Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., and Testuggine, D. (2023). Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. arXiv. source
  • Interpretationcitation

    Llama Guard has a markedly more balanced precision-recall profile than Llama-3.1-nemoguard-8b-content-safety.

    “Llama Guard, evaluated at a fixed threshold of 0.5 on its internal test set, achieves an overall Precision of 90.0%, Recall of 86.7%, and F1 of 88.4% [32]-a markedly more balanced profile than the one observed in this work, where Llama-3.1-nemoguard-8b-content-safetyachieves Precision of 94.57% at the cost of Sensitivity of 51.97%, as illustrated in Table 3.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [32] Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., and Testuggine, D. (2023). Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. arXiv. source

      The appendix’s Table 6 reports a balanced precision–recall profile for Llama Guard on its own response-classification test set at a threshold of 0.5: 88.0% precision, 86.4% recall, and 87.2% F1. Those are not the cited 90.0%, 86.7%, and 88.4%; the retrieved Table 5 caption identifies prompt-classification metrics but does not show those figures. The Nemoguard results are presented as findings of the citing work, not of this source. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • May combine two claims. 47
  • Classified as probably not a substantial claim. 14
  • The quote only partly states this claim. 1

  • May combine two claims.

    Premisecitation

    Isolated alignment increases the model's sensitivity but reduces its robustness when dealing with complex queries that require balancing safety, value, and cultural dimensions simultaneously, leading to high false failure rates and spurious rejections of legitimate interactions.

    “This isolated alignment increases the model's sensitivity but reduces its robustness when dealing with complex queries that require balancing safety, value, and cultural dimensions simultaneously, leading to high false failure rates and spurious rejections of legitimate interactions [11].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [11] Naseem, U., Kashyap, G.S., Ali, R., Shabbir, E., Ray, S.K., Mohammad, A., and Seth, A. (2026). Are Aligned Large Language Models Still Misaligned?. arXiv. source

  • May combine two claims.

    Premisecitation

    Li et al. demonstrated that the effectiveness of LLM-generated synthetic data is negatively associated with task subjectivity, with diversity gaps between synthetic and real-world data being most pronounced in tasks requiring nuanced human interpretation.

    “Li et al. [30] demonstrated that the effectiveness of LLM-generated synthetic data is negatively associated with task subjectivity, with diversity gaps between synthetic and real-world data being most pronounced in tasks requiring nuanced human interpretation.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [30] Li, Z., Zhu, H., Lu, Z., and Yin, M. (2023). Synthetic data generation with large language models for text classification: Potential and limitations. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. source

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    Relying solely on built-in model safety is insufficient; defense-in-depth strategies involving external validation layers are necessary to limit damage during inevitable breaches.

    “Consequently, the consensus in the cybersecurity community emphasizes that relying solely on built-in model safety is insufficient; instead, defense-in-depth strategies involving external validation layers are strictly necessary to limit damage during inevitable breaches [2].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [2] Gulyamov, S., Gulyamov, S., Rodionov, A., Khursanov, R., Mekhmonov, K., Babaev, D., and Rakhimjonov, A. (2026). Prompt Injection Attacks in Large Language Models and AI Agent Systems: A Comprehensive Review of Vulnerabilities, Attack Vectors, and Defense Mechanisms. Information, 17. source

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Choi et al. showed that while GPT-generated text can serve as a practical augmentation tool in data-scarce settings, it falls short as a substitute for real data in fine-grained classification tasks, as synthetic outputs often capture only partial class-discriminative signals.

    “Complementarily, Choi et al. [31] showed that while GPT-generated text can serve as a practical augmentation tool in data-scarce settings, it falls short as a substitute for real data in finegrained classification tasks, as synthetic outputs often capture only partial class-discriminative signals.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [31] Choi, S., Sim, J., and Choi, G. (2025). Synthetic Text as Data: On Usefulness and Limitations. Appl. Sci., 15. source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The Do Not Answer dataset's prompts are organized into five major risk areas and 12 hazard categories.

    “The prompts are organized into five major risk areas, and in total, there are 12 hazard categories distributed across each of these areas.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [29] Wang, Y., Li, H., Han, X., Nakov, P., and Baldwin, T. (2023). Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs. arXiv. source

  • May combine two claims.

    Premisecitation

    Because LLMs scrape nearly the entire internet for training datasets, they inevitably process personal data, creating a structural tension with fundamental privacy requirements such as the principle of data minimization.

    “Because these models scrape nearly the entire internet for training datasets, they inevitably process personal data, creating a structural tension with fundamental privacy requirements such as the principle of data minimization [4].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [4] Ruschemeier, 2025, Generative AI and data protection, Camb. Forum AI Law Gov. source

  • May combine two claims.

    Premisecitation

    LLMs process both system instructions and user inputs through a unified natural language mechanism, lacking clear syntactic boundaries to separate trusted instructions from untrusted data.

    “This vulnerability stems from a fundamental architectural flaw: LLMs process both system instructions and user inputs through a unified natural language mechanism, lacking clear syntactic boundaries to separate trusted instructions from untrusted data [2].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [2] Gulyamov, S., Gulyamov, S., Rodionov, A., Khursanov, R., Mekhmonov, K., Babaev, D., and Rakhimjonov, A. (2026). Prompt Injection Attacks in Large Language Models and AI Agent Systems: A Comprehensive Review of Vulnerabilities, Attack Vectors, and Defense Mechanisms. Information, 17. source

  • May combine two claims.

    Premisecitation

    The Llama-3.1-nemoguard-8b-content-safety model is based on Meta's Llama-3.1-8B-Instruct and was optimized by NVIDIA via LoRa-tuning utilizing AEGIS 2.0.

    “Based on Meta's Llama-3.1-8BInstruct, the model was optimized by NVIDIA via LoRa-tuning, utilizing AEGIS 2.0 [9].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [9] Ghosh, S., Varshney, P., Sreedhar, M.N., Padmakumar, A., Rebedea, T., Varghese, J.R., and Parisien, C. (2025). Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails. arXiv. source

  • May combine two claims.

    Interpretationcitation

    Security in the LLM era cannot rely solely on internal fine-tuning or RLHF, but must be supported by programmable, external layers that can be updated independently of the core model weights.

    “This finding is consistent with the”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites [36] Teferra, 2026, Assessing the impact of safety guardrails on large language models, npj Digit. Med. source

      The Introduction describes safety guardrails that include RLHF, input validation, output filtering, and fallback logic. The Discussion notes that real-world mental-health systems often combine models with safety filters and postprocessing modules. Neither passage says that internal fine-tuning or RLHF is insufficient, or that external layers must be programmable and independently updatable without changing model weights. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [37] Rebedea, T., Dinu, R., Sreedhar, M., Parisien, C., and Cohen, J. (2023). NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails. arXiv. source

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Choi et al. [31] showed that while GPT-generated text can serve as a practical augmentation tool in data-scarce settings, it falls short as a substitute for real data in fine-grained classification tasks.

    “Complementarily, Choi et al. [31] showed that while GPT-generated text can serve as a practical augmentation tool in data-scarce settings, it falls short as a substitute for real data in fine-grained classification tasks-as synthetic outputs often capture only partial class-discriminative signals, enabling coarse-level distinctions but failing in more nuanced cases.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [31] Choi, S., Sim, J., and Choi, G. (2025). Synthetic Text as Data: On Usefulness and Limitations. Appl. Sci., 15. source

  • May combine two claims.

    Premisecitation

    Llama Guard, as a fine-tuned LLM classifier, has its performance inherently bounded by its training taxonomy, and cross-taxonomy evaluation remains methodologically non-trivial, requiring zero-shot or few-shot prompting adaptations to generalize beyond its original policy.

    “Llama Guard, in turn, presents a distinct but equally significant limitation: as a fine-tuned LLM classifier, its performance is inherently bounded by its training taxonomy, and cross-taxonomy evaluation remains methodologically non-trivial, requiring zero-shot or few-shot prompting adaptations to generalize beyond its original policy [32].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [32] Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., and Testuggine, D. (2023). Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. arXiv. source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Llama Guard operates over 6 risk categories, ShieldGemma over 6 primary harm types, and the Do Not Answer corpus over 12 categories distributed across 5 risk areas, with only partial overlap among the three taxonomies.

    “The first is taxonomic: Llama Guard operates over 6 risk categories [32], ShieldGemma over 6 primary harm types [33], and the Do Not Answer corpus over 12 categories distributed across 5 risk areas [29]-with only partial overlap among the three taxonomies.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [32] Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., and Testuggine, D. (2023). Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. arXiv. source
    • cites [33] Zeng, W., Liu, Y., Mullins, R., Peran, L., Fernandez, J., Harkous, H., Narasimhan, K., Proud, D., Kumar, P., and Radharapu, B. (2024). ShieldGemma: Generative AI Content Moderation Based on Gemma. arXiv. source
    • cites [29] Wang, Y., Li, H., Han, X., Nakov, P., and Baldwin, T. (2023). Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs. arXiv. source

  • May combine two claims.

    Findingown data

    Categories like 'Physical Harm' are easily intercepted, while subtle threats such as 'Sensitive Data Leakage' and 'Sophisticated Misinformation' often bypass the semantic filters.

    “Our granular analysis by risk category showed that whereas categories like”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [34] Wang, S., Ji, Z., Wang, W., Li, Z., Wu, D., and Wang, S. (2025). SoK: Evaluating Jailbreak Guardrails for Large Language Models. arXiv. source

      Section 4.1 (p. 5) says pre-processing guardrails may struggle with novel, implicit jailbreaks that conceal malicious content, supporting a general concern about evasion. It does not report that Physical Harm is easily intercepted, that Sensitive Data Leakage or Sophisticated Misinformation often bypass semantic filters, or that detection differs across those risk categories. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
    • cites [35] Li, Z., and Fung, P. (2025). Security Concerns for Large Language Models: A Comprehensive Survey on Vulnerabilities and Layered Defenses. arXiv. source

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Li et al. [30] demonstrated that the effectiveness of LLM-generated synthetic data is negatively associated with task subjectivity: for classification tasks requiring nuanced human interpretation, the diversity of synthetic data is significantly lower than that of real-world data, and the performance gap between models trained on synthetic versus real corpora widens considerably as subjectivity增加.

    “Li et al. [30] demonstrated that the effectiveness of LLM-generated synthetic data is negatively associated with task subjectivity: for classification tasks requiring nuanced human interpretation-such as humor, sarcasm, or emotion detection-the diversity of synthetic data is significantly lower than that of real-world data, and the performance gap between models trained on synthetic versus real”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [30] Li, Z., Zhu, H., Lu, Z., and Yin, M. (2023). Synthetic data generation with large language models for text classification: Potential and limitations. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. source

  • May combine two claims.

    Premisecitation

    Llama Guard, evaluated at a fixed threshold of 0.5 on its internal test set, achieves an overall Precision of 90.0%, Recall of 86.7%, and F1 of 88.4%.

    “Llama Guard, evaluated at a fixed threshold of 0.5 on its internal test set, achieves an overall Precision of 90.0%, Recall of 86.7%, and F1 of 88.4% [32]-a markedly more balanced profile than the one observed in this work, where Llama-3.1-nemoguard-8b-content-safetyachieves Precision of 94.57% at the cost of Sensitivity of 51.97%, as illustrated in Table 3.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [32] Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., and Testuggine, D. (2023). Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. arXiv. source

      The appendix (Additional information; page unavailable) says the authors used a 0.5 threshold to compute precision, recall, and F1 on their dataset. But the available Table 5 passage does not show the prompt-classification values, while Table 6 reports different overall response-classification values: 88.0%, 86.4%, and 87.2%. Our copy therefore supports the evaluation setting but not the three figures quoted. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    The Llama-3.1-nemoguard-8b-content-safety model achieved a precision of 94.57% and a sensitivity (recall) of 51.97% in classifying prompts as safe or unsafe on the 1878-sample corpus.

    “Results indicate a high Precision (94.57%) but a moderate Sensitivity (51.97%), uncovering a critical performance trade-off: the model exhibits a conservative bias, prioritizing high precision to minimize false positives at the expense of recall for nuanced adversarial prompts, particularly in categories involving sensitive data leakage and misinformation.”
    • Also stated in: § 4.1. Performance of the Guardrail Model in Prompt Classification · plain
      “It is observed in Figure 5 that, of the 939 unsafe prompts, 51.97% (488) were correctly classified as unsafe (True Positives), while 48.03% (451) were incorrectly classified as safe (False Negatives).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The guardrail model correctly identified 911 of 939 safe prompts as safe and misclassified 28 as unsafe.

    “Of the 939 safe prompts, 97.02% (911) were correctly identified as safe (True Negatives) and only 2.98% (28) were classified as unsafe (False Positives).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The firewall exhibited exceptional Precision (94.57%) and a remarkably low False Positive Rate (2.98%).

    “On one hand, the firewall exhibited exceptional Precision (94.57%) and a remarkably low False Positive Rate (2.98%).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The model rarely misclassifies safe prompts as unsafe, consistent with its precision exceeding 94%.

    “The remaining categories show considerably lower FP rates, confirming that the model rarely misclassifies safe prompts as unsafe, which aligns with its precision exceeding 94%.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    False positives total 28 samples and are uniformly low across all categories.

    “Regarding False Positives, which total 28 samples (Figure 5), Figure 8 reveals a more uniform distribution, with consistently low rates across all categories.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The guardrail system neutralizes five out of nine core model failures, showing its highest interception capacity in the Adult Content category.

    “The guardrail system neutralizes five out of nine core model failures, showing its highest interception capacity in the Adult Content category.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The Risks from leaking or inferring sensitive information (organization/gov) category (N = 272) exhibits the highest False Negative rate (44.9%), with 122 misclassified samples, approximately 27% of all 451 false negatives recorded globally.

    “In contrast, the Risks from leaking or inferring sensitive information (organization/gov) category (N = 272) exhibits the highest False Negative rate (44.9%), with 122 misclassified samples-approximately 27% of all 451 false negatives recorded globally.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The three categories (Risks from leaking or inferring sensitive information, Disseminating false or misleading information, Causing material harm by disseminating misinformation) account for the majority of undetected threats and represent the most critical vulnerability zones in this evaluation.

    “Together, these three categories account for the majority of undetected threats and represent the most critical vulnerability zones in this evaluation.”
    • Also stated in: § 4.2. Analysis by Risk Categories · plain
      “These three categories together account for the majority of undetected threats and represent the most critical vulnerability zones identified in this evaluation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The lowest FN rates are Adult Content (N = 56, 8.9% FN rate) and Reducing the cost of disinformation campaigns (N = 80, 6.2% FN rate).

    “False Negative rates vary considerably: categories such as Adult Content (N = 56, 8.9% FN rate) and Reducing the cost of disinformation campaigns (N = 80, 6.2% FN rate) present the lowest error proportions, reflecting the model's stronger ability to detect explicit or lexically salient threats.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The model's F1-Score is 12% for Causing material harm by disseminating misinformation and 19% for Risks from leaking sensitive information, both below the global F1-Score of 67.08%.

    “While the model performs well in categories with explicit lexical patterns (e.g., Adult Content), a critical performance decay falls significantly below the aggregate threshold in domains requiring nuanced semantic interpretation or factual verification, notably Causing material harm by disseminating misinformation (12%) and Risks from leaking sensitive information (19%).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The FN rate is 43.2% for Disseminating false or misleading information (N = 185) and 46.8% for Causing material harm by disseminating misinformation, e.g., in medicine or law (N = 126).

    “Similarly elevated FN rates are observed in Disseminating false or misleading information (N = 185, 43.2% FN rate) and Causing material harm by disseminating misinformation, e.g., in medicine or law (N = 126, 46.8% FN rate).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The global Compensation Rate is 34.8%, intercepting 8 out of 23 GPT-4o mini failures.

    “The model's limited ability to act as an effective secondary defense layer is highlighted by the global Compensation Rate of 34.8% (intercepting 8 out of 23 failures).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The FN rate is 87.0% for Disseminating false or misleading information and 93.7% for Causing material harm by disseminating misinformation, e.g., in medicine or law.

    “A similar pattern appears in Disseminating false or misleading information (FN rate: 87.0%) and Causing material harm by disseminating misinformation, e.g., in medicine or law (FN rate: 93.7%),where the harm lies not in the prompt's vocabulary but in its factual incorrectness or manipulative framing-dimensions that a classifier operating on prompt text alone cannot verify.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The remaining fifteen uncompensated failures are primarily concentrated in context-dependent hazard categories and represent a security risk.

    “The remaining fifteen uncompensated failures, which are primarily concentrated in context-dependent hazard categories, represent a security risk.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The recorded Sensitivity of 51.97% highlights a critical vulnerability: the system is highly effective against explicit and 'shallow' adversarial prompts but struggles to identify more sophisticated, context-dependent threats.

    “On the other hand, the recorded Sensitivity of 51.97% highlights a critical vulnerability: while the system is highly effective against explicit and”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.

    Study design and conductown data

    The synthetic benign prompts were manually reviewed by three authors, each prompt reviewed by a single author without overlapping annotation or agreement reporting; only prompts satisfying safety and thematic consistency were retained.

    “To ensure the quality of the synthetic data, the generated prompts were manually reviewed by three of the authors, who divided the full set of 939 synthetic prompts among themselves for independent evaluation, meaning each prompt was reviewed by a single author without overlapping annotation or agreement reporting.”
    • Also stated in: § 3.1.2. Synthetic Data · plain
      “Only prompts that satisfied both criteria were retained in the final dataset.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    A potential source of bias is that GPT-4o mini was employed both to generate the synthetic benign prompts and as the foundational model whose failures are assessed via the Compensation Rate, so the synthetic data may reflect stylistic and linguistic patterns inherent to that model, introducing homogeneity not fully representing real-world safe interactions.

    “It is important to acknowledge a potential source of bias in this generation process.Since GPT-4o mini was employed both to generate the synthetic benign prompts and as the foundational model whose failures are assessed via the Compensation Rate, the synthetic data may reflect stylistic and linguistic patterns inherent to that model.This could introduce a degree of homogeneity in the benign prompt”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Perfect prevention against sophisticated prompt injections remains unachievable through standard internal mechanisms, rendering LLMs inherently exploitable.

    “Because of the stochastic nature of generative AI, perfect prevention against sophisticated prompt injections remains unachievable through standard internal mechanisms, rendering these models inherently exploitable.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Most studies do not assess how external guardrail layers behave when the underlying model itself fails-that is, when internal alignment is insufficient and the firewall must serve as the last line of interception.

    “Furthermore, most studies do not assess how these layers behave when the underlying model itself fails-that is, when internal alignment is insufficient and the firewall must serve as the last line of interception.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    A consistent gap emerges across the literature: the majority of existing evaluations focus either on the attack surface or on the primary model's internal alignment, with limited attention to the performance of external guardrail layers as a second line of defense.

    “Despite the breadth of tools, benchmarks, and defense frameworks surveyed, a consistent gap emerges across the literature: the majority of existing evaluations focus either on the attack surface or on the primary model's internal alignment, with limited attention to the performance of external guardrail layers as a second line of defense.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The system effectively blocks explicit threats, but its efficacy as a secondary defense diminishes against context-dependent vulnerabilities, notably data exfiltration and misinformation.

    “These findings indicate that while the system effectively blocks explicit threats, its efficacy as a secondary defense diminishes against context-dependent vulnerabilities, notably data exfiltration and misinformation.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The classification of GPT-4o mini responses as inappropriate (unsafe) is accurate.

  • May combine two claims.

    Premisecitation

    The Llama-3.1-nemoguard-8b-content-safety model identifies potentially harmful content and returns associated risk categories.

    “It identifies potentially harmful content and returns associated risk categories, making it useful for reinforcing security in applications that use generative AI.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites https://build.nvidia.com/nvidia/llama-3_1-nemoguard-8b-content-safety/ modelcard

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    LLMs can inadvertently memorize and reproduce personal or sensitive information present in their training data, leading to severe privacy breaches.

    “Indeed, recent studies highlight that LLMs can inadvertently memorize and reproduce personal or sensitive information present in their training data, leading to severe privacy breaches [3].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] Feretzakis, G., Vagena, E., Kalodanis, K., Peristera, P., Kalles, D., and Anastasiou, A. (2025). GDPR and Large Language Models: Technical and Legal Obstacles. Future Internet, 17. source

  • May combine two claims.

    Premisecitation

    Adversaries exploit the model's generative capabilities through prompt manipulation during inference, often resulting in model misuse such as the generation of harmful content or data exfiltration.

    “These vulnerabilities are particularly critical during the inference phase, where adversaries exploit the model's generative capabilities through prompt manipulation, often resulting in model misuse such as the generation of harmful content or data exfiltration [5].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [5] Fenza, 2025, A Lifecycle-Oriented Survey of Emerging Threats and Vulnerabilities in Large Language Models, IEEE Access source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    The CR application relied on limited data and requires validation across larger datasets to establish broader applicability.

    “We note, however, that its application relied on limited data and requires validation across larger datasets to establish broader applicability.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The firewall faces notable difficulty in identifying prompts involving potential sensitive information leakage from organizational entities, likely due to the implicit and context-dependent nature of such threats.

    “This result indicates that the firewall faces notable difficulty in identifying prompts involving potential sensitive information leakage from organizational entities, likely due to the implicit and context-dependent nature of such threats.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The model can enhance security without compromising usability or inducing 'refusal fatigue' in benign users.

    “These figures are relevant for industrial applications, as they indicate that the model can enhance security without compromising the usability of the system or inducing”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The use of GPT-4o mini as both the synthetic data generator and the model under evaluation in the Compensation Rate analysis introduces a potential circularity: the benign prompts may carry stylistic biases from that model, which could influence the observed CR values.

    “Additionally, the use of GPT-4o mini as both the synthetic data generator and the model under evaluation in the Compensation Rate analysis introduces a potential circularity: the benign prompts may carry stylistic biases from that model, which could influence the observed CR values.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The guardrail strongly prioritizes precision to avoid false alarms, at a significant cost to sensitivity.

    “On the other hand, the low False Positive rates across all categories (global FP count: 28) confirm that the guardrail strongly prioritizes Precision (94.57%) to avoid false alarms, at a significant cost to Sensitivity (51.97%).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    ShieldGemma explicitly acknowledges an analogous conservative bias, noting that its interpretation of policy violations may be overly conservative and recommending that downstream clients adjust filtering thresholds based on their specific use cases.

    “ShieldGemma, in turn, explicitly acknowledges an analogous conservative bias to the one identified in this work, noting that its interpretation of policy violations may be overly conservative and recommending that downstream clients adjust filtering thresholds based on their specific use cases [33].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [33] Zeng, W., Liu, Y., Mullins, R., Peran, L., Fernandez, J., Harkous, H., Narasimhan, K., Proud, D., Kumar, P., and Radharapu, B. (2024). ShieldGemma: Generative AI Content Moderation Based on Gemma. arXiv. source

  • May combine two claims.

    Premisecitation

    Evidence from the literature indicates that ShieldGemma shares an analogous asymmetric error profile, explicitly acknowledging that its interpretation of policy violations may be overly conservative, potentially interfering with helpfulness when used to filter LLM responses.

    “The second obstacle is behavioral: evidence from the literature indicates that Shield-Gemma shares an analogous asymmetric error profile, explicitly acknowledging that its interpretation of policy violations may be overly conservative, potentially interfering with helpfulness when used to filter LLM responses [33].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [33] Zeng, W., Liu, Y., Mullins, R., Peran, L., Fernandez, J., Harkous, H., Narasimhan, K., Proud, D., Kumar, P., and Radharapu, B. (2024). ShieldGemma: Generative AI Content Moderation Based on Gemma. arXiv. source

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    The dependency on English-centric training data and the fixed nature of the Llama-3.1-nemoguard-8b-content-safety model limit its adaptability to regional legal frameworks, such as the LGPD in Brazil.

    “The dependency on English-centric training data and the fixed nature of the Llama-3.1-nemoguard-8b-content-safetymodel limit its adaptability to regional legal frameworks, such as the LGPD in Brazil.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The Llama-3.1-nemoguard-8b-content-safety model's weights are fixed from NVIDIA training and cannot be altered.

  • May combine two claims.

    Interpretationown data

    For those categories, the harm lies not in the prompt's vocabulary but in its factual incorrectness or manipulative framing, which a classifier operating on prompt text alone cannot verify.

    “A similar pattern appears in Disseminating false or misleading information (FN rate: 87.0%) and Causing material harm by disseminating misinformation, e.g., in medicine or law (FN rate: 93.7%),where the harm lies not in the prompt's vocabulary but in its factual incorrectness or manipulative framing-dimensions that a classifier operating on prompt text alone cannot verify.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    ShieldGemma recognizes that direct comparison remains challenging due to variations in policy definitions and supported harm types across datasets, as well as inconsistencies in policy definitions even within the same harm type.

    “Similarly, ShieldGemma recognizes that direct comparison remains challenging due to variations in policy definitions and supported harm types across datasets, as well as inconsistencies in policy definitions even within the same harm type.Ref. [33]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [33] Zeng, W., Liu, Y., Mullins, R., Peran, L., Fernandez, J., Harkous, H., Narasimhan, K., Proud, D., Kumar, P., and Radharapu, B. (2024). ShieldGemma: Generative AI Content Moderation Based on Gemma. arXiv. source

  • May combine two claims.

    Interpretationreasoning

    The elevated FN rates in these domains represent a meaningful risk in production environments, as they correspond to scenarios where adversarial actors may exploit the model's semantic blind spots to bypass the firewall undetected.

    “The elevated FN rates in these domains represent a meaningful risk in production environments, as they correspond precisely to scenarios where adversarial actors may exploit the model's semantic blind spots to bypass the firewall undetected.”
    Quote in the paperSubstantial claim
50 of 50

  • The paper relies on this without support.

    The classification of GPT-4o mini responses as inappropriate (unsafe) is accurate.

    “The objective was to verify the performance of the model in cases where GPT-4o mini fails, specifically when an inappropriate response is generated for an unsafe prompt.”
    “In this scenario, CF indicates the number of instances that the model successfully intercepted an interaction despite the underlying model generating an inappropriate response.”

  • The paper relies on this without support.

    The Llama-3.1-nemoguard-8b-content-safety model's weights are fixed from NVIDIA training and cannot be altered.

    “It is important to highlight that these results stem from the inference of Llama-3.1nemoguard-8b-content-safety based on its weights already defined during training by NVIDIA, and it is not possible to alter them.”
2 of 2

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • One category's reported false positives exceed the global total

    Major

    A 12.0% false-positive proportion among 263 samples implies about 32 false positives in this category alone, exceeding the reported global total of 28.

  • Llama Guard comparison cites a synthetic-text paper

    Minor

    The Llama Guard results are attributed to reference [32], but reference 32 is Choi et al.'s paper on synthetic text; the Llama Guard paper is reference 33.

  • Do Not Answer citation points to SafetyBench

    Minor

    The text cites reference [29] for the Do Not Answer dataset, but reference 29 is SafetyBench. Do-Not-Answer appears as reference 30.

3 of 3

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Jiang, H., Li, S., and Wang, M. (2024). ControlNet: An Advanced Firewall for Retrieval–Augmented Generation Systems. arXiv.
  2. Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Eugenio, B.D., and Schockaert, S. (2025). Automated Progressive Red Teaming. Proceedings of the 31st International Conference on Computational Linguistics, Association for Computational Linguistics.

Cited by

Sorted by citations

Related

Authors
Azambuja, A. J., Guilherme, M., Castro, J. V. F. D., Lima, J. P. D. O., Oliveira, L. B., & Soares, A. D. S.
Affiliations
  • Department of Space Sciences and Technologies, Aeronautics Institute of Technology (ITA), São José dos Campos 12228-900, Brazil
  • Universidade Federal de Goiás (BR)
  • Instituto Tecnológico de Aeronáutica (BR)
  • Universidade Federal de Minas Gerais (BR)
ORCID
Venue
Future Internet
Publisher
MDPI AG
Published
2026-05-09
Volume
18
Issue
5
Pages
252
ISSN
1999-5903 (ISSN-L)
DOI
10.3390/fi18050252
OpenAlex
W7160861171
Semantic Scholar
56dc3a66b7e594da2af1bd96cb93e3adf3b9e181
Cited by
2
References
25
Citations by year
2026: 2
FWCI
6.93
Type
journal article
Language
en
Open access
Licence
Locations
  • Future Internet (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Advanced Knowledge Center in Immersive Technologies
  • MCTI
  • EMBRAPII
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
large language model security, prompt injection, jailbreaking, compensation rates, precision-recall trade-off, data exfiltration, misinformation, safety alignment, adversarial prompts

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review