NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
-
Updated
Jul 17, 2026 - Python
#
llm-safety
Here are 276 public repositories matching this topic...
DeepTeam is a framework to red team LLMs and AI agents.
-
Updated
Jul 8, 2026 - Python
UQLM: Uncertainty Quantification for Language Models, is a Python package for UQ-based LLM hallucination detection
uncertainty-quantification uncertainty-estimation ai-safety confidence-score hallucination confidence-estimation ai-evaluation llm llm-evaluation llm-safety hallucination-evaluation hallucination-detection hallucination-mitigation llm-hallucination
-
Updated
Jul 9, 2026 - Python
A resource repository for machine unlearning in large language models
nlp machine-learning awesome privacy evaluation alignment awesome-list ai-safety right-to-be-forgotten machine-unlearning paper-list responsible-ai unlearning llm model-editing large-language-model llm-safety llm-unlearning selective-forgetting knowledge-erasure
-
Updated
Jul 14, 2026
Agent trace and tool-use safety evaluation lab.
-
Updated
May 2, 2026 - Python
Decrypted Generative Model safety files for Apple Intelligence containing filters
-
Updated
Jan 26, 2026 - Python
Static security scanner for LLM agents — prompt injection, MCP config auditing, taint analysis. 51 rules mapped to OWASP Agentic Top 10 (2026). Works with LangChain, CrewAI, AutoGen.
python cli security mcp static-analysis owasp taint-analysis vulnerability-detection vulnerability-scanner ai-security ai-agent langchain prompt-injection llm-security llm-safety crewai ai-security-tool langchain-security-
-
Updated
Jul 4, 2026 - Python
[NDSS'25 Best Technical Poster] A collection of automated evaluators for assessing jailbreak attempts.
-
Updated
Apr 1, 2025 - Python
Papers about red teaming LLMs and Multimodal models.
-
Updated
Jul 14, 2026
Attack to induce LLMs within hallucinations
-
Updated
May 17, 2024 - Python
Open Source Reliability Harness: Make your agents follow rules. One line of code to enforce, trace, and improve.
rule-engine ai-safety policy-engine ai-security guardrails responsible-ai ai-policy ai-compliance ai-governance prompt-injection llm-security llm-proxy llm-safety content-safety llm-monitoring llm-guardrails agentic-ai ai-guardrails ai-firewall ai-audit
-
Updated
May 23, 2026 - Python
Reading list for adversarial perspective and robustness in deep reinforcement learning.
deep-reinforcement-learning ai-safety machine-learning-security ai-security robust-machine-learning ai-alignment safe-reinforcement-learning robust-reinforcement-learning responsible-ai adversarial-reinforcement-learning secure-ai ml-security llm-security reinforcement-learning-safety llm-safety artificial-intelligence-security artificial-intelligence-alignment robust-deep-reinforcement-learning reinforcement-learning-security large-language-model-safety
-
Updated
Mar 2, 2026
Alignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface
python security benchmark machine-learning ai jailbreak evaluation santander alignment ai-safety red-teaming content-moderation guardrails responsible-ai llm prompt-injection llm-safety autoresearch
-
Updated
Jul 16, 2026 - Python
Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses | 500+ Papers | Perception, Cognition, Planning, Interaction, Agentic System
robotics jailbreak survey awesome-list autonomous-driving ai-safety adversarial-attacks red-teaming backdoor-attacks adversarial-robustness embodied-ai trustworthy-ai large-language-models prompt-injection world-model llm-safety vision-language-action embodied-agents agent-safety world-action-model
-
Updated
Jul 18, 2026 - Python
[NeurIPS 2025] SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations
-
Updated
Dec 10, 2025 - Python
Official repository for the paper "ALERT: A Comprehensive Benchmark for Assessing Large Language Models’ Safety through Red Teaming"
nlp benchmark ai artificial-intelligence nlp-machine-learning red-teaming bias-detection safety-monitoring transformers-models llm llm-evaluation llm-safety llm-safety-benchmark
-
Updated
Sep 20, 2024 - Python
A deterministic verification layer for AI systems. QWED verifies AI outputs using mathematics, symbolic reasoning, and formal methods (Z3, SMT, SymPy), creating an auditable trust boundary for agentic AI. Not generation. Verification.
python nlp machine-learning sympy smt-solver formal-verification ai-safety ai-security hallucination code-security ai-accuracy enterprise-ai neurosymbolic-ai llm-safety hallucination-detection z3-prover deterministic-ai llm-verification deterministic-verification deterministic-verification-layer
-
Updated
Jul 15, 2026 - Python
lintlang is a static linter for AI agent configs, tool descriptions, and system prompts that runs zero-LLM quality gating in CI. Catches language-level failures (vague tool descriptions, missing stop conditions, schema gaps) before they reach runtime, with deterministic regex + structural detectors and no model calls.
linter static-analysis ci-cd developer-tools ai-safety ai-agents quality-gate llm prompt-engineering llm-ops llm-safety system-prompts ai-audit ai-reliability agent-config prompt-linter tool-descriptions zero-llm hermes-labs agent-linter
-
Updated
Jul 18, 2026 - Python
Papers from our SoK on Red-Teaming (Accepted at TMLR)
-
Updated
Jun 17, 2026
NeurIPS'24 - LLM Safety Landscape
-
Updated
Oct 21, 2025 - Python
Improve this page
Add a description, image, and links to the llm-safety topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-safety topic, visit your repo's landing page and select "manage topics."