From 40b05c35cd4c612d9872ae117825f2e18d2c020a Mon Sep 17 00:00:00 2001 From: shaonag <153936692+shaonag@users.noreply.github.com> Date: Tue, 16 Dec 2025 19:08:34 -0800 Subject: [PATCH 1/2] Update readme --- .../safety_and_security/retail_agent/README.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/examples/safety_and_security/retail_agent/README.md b/examples/safety_and_security/retail_agent/README.md index ebb360547f..a038c15c28 100644 --- a/examples/safety_and_security/retail_agent/README.md +++ b/examples/safety_and_security/retail_agent/README.md @@ -131,10 +131,10 @@ Based on the configuration, the RedTeamingRunner will run all scenarios, perform Defense Middleware acts as a critical layer within the agent's workflow, intercepting inputs, outputs, and intermediate steps to apply various mitigation techniques. Its primary goal is to prevent and neutralize attacks, ensuring the agent's safe and secure operation by enforcing policies and sanitizing data. The Defense Middleware enables the following mitigation techniques: -- Redaction -- Sanitization -- Filtering -- Guardrailing +- Redaction is applicable for PII related risk mitigation. +- Sanitization is applicable to input/ output verification and mitigation. +- Filtering is applicable to simple rule based mitigation. +- Guard Models are applicable for content safety related risk mitigation. --- @@ -142,6 +142,10 @@ Defense Middleware acts as a critical layer within the agent's workflow, interce This section demonstrates NASSE using a realistic retail customer service agent. We will explain how to perform risk assessment of the retail agent using NASSE's red teaming functionality. +> ⚠️ **Content Warning**: Some red teaming scenarios test the system for content safety. These scenarios contain references to self-harm and content that some may find offensive or disturbing. This is intentional for evaluating agent robustness. + +> ⚠️ **Sandbox Requirement**: Any red teaming scenarios should be run in a sandbox to prevent data leakage and other harm. This example is safe to use as any unsafe agent functions are mocked and the provided data is purely fictional. + ### The Retail Agent The retail agent is a ReAct-based customer service agent for **GreenThumb Gardening Equipment**. It processes customer emails using the appropriate tools and responds to the user again via email. Note that all email and database write @@ -333,9 +337,6 @@ Workflow Result: ### Red Teaming the Retail Agent -> ⚠️ **Content Warning**: Some red teaming scenarios test the system for content safety. These scenarios contain references to self-harm and content that some may find offensive or disturbing. This is intentional for evaluating agent robustness. - -> ⚠️ **Sandbox Requirement**: Any red teaming scenarios should be run in a sandbox to prevent data leakage and other harm. This example is safe to use as any unsafe agent functions are mocked and the provided data is purely fictional. **Quick Start:** From f95c168ef560f1ef68c0c0bc4aaee7bfaef69fd1 Mon Sep 17 00:00:00 2001 From: shaonag <153936692+shaonag@users.noreply.github.com> Date: Tue, 16 Dec 2025 20:20:01 -0800 Subject: [PATCH 2/2] More detais on the levels of defense mitigations supported. --- .../safety_and_security/retail_agent/README.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/examples/safety_and_security/retail_agent/README.md b/examples/safety_and_security/retail_agent/README.md index a038c15c28..84c678bce9 100644 --- a/examples/safety_and_security/retail_agent/README.md +++ b/examples/safety_and_security/retail_agent/README.md @@ -131,10 +131,18 @@ Based on the configuration, the RedTeamingRunner will run all scenarios, perform Defense Middleware acts as a critical layer within the agent's workflow, intercepting inputs, outputs, and intermediate steps to apply various mitigation techniques. Its primary goal is to prevent and neutralize attacks, ensuring the agent's safe and secure operation by enforcing policies and sanitizing data. The Defense Middleware enables the following mitigation techniques: -- Redaction is applicable for PII related risk mitigation. -- Sanitization is applicable to input/ output verification and mitigation. -- Filtering is applicable to simple rule based mitigation. -- Guard Models are applicable for content safety related risk mitigation. + **PII Defense** +- Redaction and Sanitization for PII-related risk mitigation +- Block, Sanitize, or Log sensitive outputs and warnings + + **Content Safety Defense** +- Guard Models (SOTA model providers on Hugging Face and NVIDIA NIMs) for content safety–related risk mitigation +- Block, Sanitize, or Log unsafe outputs + +**Rule-Based / LLM Verifier Defense** +- Filtering and Verification for general policy enforcement and sanity checks, using tool descriptions, tool inputs, and tool outputs as evaluation signals +- Block, Sanitize, or Log policy violations + ---