Nemo Agent Toolkit Safety and Security Engine PR description

[shaonag]

This PR introduces NASSE (NeMo Agent Safety and Security Engine) and a full Retail Agent example that demonstrates how to proactively assess and improve an agent’s safety/security posture via configurable red teaming and defense middleware, thus realizing the first implementation of the framework proposed in A Safety and Security Framework for Real World Agentic Systems https://www.arxiv.org/abs/2511.21990.

What This PR Adds

NASSE Red Teaming Framework

End-to-end tooling to instrument agent workflows, execute attack scenarios, and automatically evaluate + report risk:

• Attack injection into registered function inputs/outputs (including nested fields via JSONPath)
• Scenario-based testing to isolate and understand “weak links” across workflow steps (not just final output)
• Large-scale evaluation runs driven by YAML and datasets, with configurable repetitions to account for non-determinism
• Portable HTML reporting with overall risk score, attack success rate, per-scenario breakdown, and filtering by safety categories /groups
• Integrated defenses and mitigations support for securing the agentic system.

RedTeamingMiddleware

Workflow interceptor to inject adversarial payloads during execution:

• Target specific functions or function groups (or <workflow>)
• Inject at input or output with configurable placement (replace / append variants)
• Modify specific schema fields using JSONPath
• Control delivery via call limits and field-resolution strategies

RedTeamingEvaluator

LLM-judge based evaluation of attack success:

• Configurable judge prompt and scenario-specific instructions
• Evaluate at specific workflow points (e.g., overall workflow output or intermediate function outputs)
• Returns 0.0–1.0 score with reasoning with support for reduction strategies when multiple steps match

RedTeamingRunner

YAML-driven orchestration of scenario suites:

• Runs all scenarios (middleware + evaluator + metadata)
• Produces a detailed HTML report summarizing quantitative findings for analysis

Defense Middleware

Pluggable mitigation layer to harden agent workflows by intercepting inputs/outputs/steps:

• PII Defense: detect and redact/sanitize sensitive entities (optional third party dependencies)
• Content Safety Guard: guard-model checks (NIM or Hugging Face) to block/sanitize/log unsafe content
• Rule-based / LLM Output Verifier: policy enforcement and verification using tool descriptions/inputs/outputs
• Supports action modes: redirection, refusal, partial_refusal
• Can be applied at workflow, function group, or specific function scope

Retail Agent Example

A realistic ReAct-style customer service agent (“GreenThumb Gardening Equipment”) used to demonstrate NASSE:

• Mocked email/database operations for safe sandbox execution
• Includes runnable configs for:
  • base workflow execution
  • red teaming runs and report generation
  • defended workflow configuration
  • red teaming against the defended workflow for before/after comparison

Optional Dependencies

• Hugging Face content safety guard model support (.[huggingface])
• NVIDIA NIM content safety guard model support
• PII defense support via Presidio (.[pii-defense])