NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
-
Updated
Aug 7, 2026 - Python
#
llm-safety
Here are 303 public repositories matching this topic...
DeepTeam is a framework to red team LLMs and AI agents.
-
Updated
Aug 5, 2026 - Python
[JMLR 2026] "UQLM: A Python Package for Uncertainty Quantification in Large Language Models"
uncertainty-quantification uncertainty-estimation ai-safety confidence-score hallucination confidence-estimation ai-evaluation llm llm-evaluation llm-safety hallucination-evaluation hallucination-detection hallucination-mitigation llm-hallucination
-
Updated
Aug 3, 2026 - Python
A resource repository for machine unlearning in large language models
nlp machine-learning awesome privacy evaluation alignment awesome-list ai-safety right-to-be-forgotten machine-unlearning paper-list responsible-ai unlearning llm model-editing large-language-model llm-safety llm-unlearning selective-forgetting knowledge-erasure
-
Updated
Aug 6, 2026
Agent trace and tool-use safety evaluation lab.
-
Updated
May 2, 2026 - Python
Decrypted Generative Model safety files for Apple Intelligence containing filters
-
Updated
Jul 19, 2026 - Python
Static security scanner for LLM agents — prompt injection, MCP config auditing, taint analysis. 51 rules mapped to OWASP Agentic Top 10 (2026). Works with LangChain, CrewAI, AutoGen.
python cli security mcp static-analysis owasp taint-analysis vulnerability-detection vulnerability-scanner ai-security ai-agent langchain prompt-injection llm-security llm-safety crewai ai-security-tool langchain-security-
-
Updated
Jul 4, 2026 - Python
[NDSS'25 Best Technical Poster] A collection of automated evaluators for assessing jailbreak attempts.
-
Updated
Apr 1, 2025 - Python
Papers about red teaming LLMs and Multimodal models.
-
Updated
Jul 14, 2026
Attack to induce LLMs within hallucinations
-
Updated
May 17, 2024 - Python
Open Source Reliability Harness: Make your agents follow rules. One line of code to enforce, trace, and improve.
rule-engine ai-safety policy-engine ai-security guardrails responsible-ai ai-policy ai-compliance ai-governance prompt-injection llm-security llm-proxy llm-safety content-safety llm-monitoring llm-guardrails agentic-ai ai-guardrails ai-firewall ai-audit
-
Updated
May 23, 2026 - Python
Reading list for adversarial perspective and robustness in deep reinforcement learning.
deep-reinforcement-learning ai-safety machine-learning-security ai-security robust-machine-learning ai-alignment safe-reinforcement-learning robust-reinforcement-learning responsible-ai adversarial-reinforcement-learning secure-ai ml-security llm-security reinforcement-learning-safety llm-safety artificial-intelligence-security artificial-intelligence-alignment robust-deep-reinforcement-learning reinforcement-learning-security large-language-model-safety
-
Updated
Mar 2, 2026
Alignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface
python security benchmark machine-learning ai jailbreak evaluation santander alignment ai-safety red-teaming content-moderation guardrails responsible-ai llm prompt-injection llm-safety autoresearch
-
Updated
Aug 1, 2026 - Python
Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses | 500+ Papers | Perception, Cognition, Planning, Interaction, Agentic System
robotics jailbreak survey awesome-list autonomous-driving ai-safety adversarial-attacks red-teaming backdoor-attacks adversarial-robustness embodied-ai trustworthy-ai large-language-models prompt-injection world-model llm-safety vision-language-action embodied-agents agent-safety world-action-model
-
Updated
Aug 8, 2026 - Python
Offline prompt-contract auditor for Hermes, Claude Code, Codex, OpenCode & OpenClaw. Pre-write guard before agents ship vague code. Zero deps. No model API.
cli devops offline static-analysis opencode hermes codex prompts sarif ai-agents zero-dependencies guardrails prompt-engineering llm-safety agent-skills claude-code openclaw hermes-agent prompt-linter pre-write-guard
-
Updated
Jul 22, 2026 - Python
[NeurIPS 2025] SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations
-
Updated
Dec 10, 2025 - Python
A curated, continuously updated reading list of 200+ papers on LLM agents: planning, memory, tool use, multi-agent, evaluation & safety. Companion to the survey 'LLM Agents: A Survey'.
awesome memory survey planning awesome-list agents multi-agent-systems autonomous-agents ai-safety ai-agents paper-list tool-use large-language-models llm prompt-injection llm-safety llm-agents agent-benchmark agent-survey
-
Updated
Aug 8, 2026 - Python
Official repository for the paper "ALERT: A Comprehensive Benchmark for Assessing Large Language Models’ Safety through Red Teaming"
nlp benchmark ai artificial-intelligence nlp-machine-learning red-teaming bias-detection safety-monitoring transformers-models llm llm-evaluation llm-safety llm-safety-benchmark
-
Updated
Sep 20, 2024 - Python
A deterministic verification layer for AI systems. QWED verifies AI outputs using mathematics, symbolic reasoning, and formal methods (Z3, SMT, SymPy), creating an auditable trust boundary for agentic AI. Not generation. Verification.
python nlp machine-learning sympy smt-solver formal-verification ai-safety ai-security hallucination code-security ai-accuracy enterprise-ai neurosymbolic-ai llm-safety hallucination-detection z3-prover deterministic-ai llm-verification deterministic-verification deterministic-verification-layer
-
Updated
Aug 9, 2026 - Python
Static analysis for AI agent configs, tool descriptions, and system prompts — catches vague tool descriptions, missing stop conditions, and schema gaps before they reach runtime. Zero-LLM, deterministic checks, built for CI.
linter static-analysis ci-cd developer-tools ai-safety ai-agents quality-gate llm prompt-engineering llm-ops llm-safety system-prompts ai-audit ai-reliability agent-config prompt-linter tool-descriptions zero-llm hermes-labs agent-linter
-
Updated
Aug 6, 2026 - Python
Improve this page
Add a description, image, and links to the llm-safety topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-safety topic, visit your repo's landing page and select "manage topics."