Multi-Provider AI Guardrails

[shaneutt]

Extend the guardrails filter to support external content safety providers and response-side inspection.

If feasible make this generic, so that we can plug into a specific provider now (say, Nemo), but keep the schema generalized to support others easily later.

Scope:

• External guardrail provider integration via webhook callout
• Built-in PII detection (SSN, credit card, phone, email, URL patterns)
• Response-side guardrails (inspect LLM output before returning to client)
• Actions: allow, reject, mask (anonymize matched content)
• Per-route guardrail policy configuration
• Configurable fail-open/fail-closed on provider errors

[liavweiss]

Hi @christinaexyou,

I'd love to join the effort here! I was thinking about how we can split the work
to move in parallel. Here's how I see the dependencies between the sub-tasks:

1. Fail-open/fail-closed - prerequisite for external providers
2. External provider integration - provider trait + webhook callout (blocked by 1)
3. Response-side guardrails - reuses provider infrastructure (blocked by 2)
4. Actions: mask/anonymize - needs provider or PII to return modified text (blocked by 2)
5. Per-route policy - mostly independent, can be done anytime
6. Built-in PII detection - @christinaexyou you're on this, right?

I have some experience with external guardrail providers implementaion (Nemo, Bedrock) from
a previous project, so happy to pick up (1) and (2) if that works for you? (I will open a separate sub-issue for the external provider integration).

Let me know what you think!

[christinaexyou]

@liavweiss sounds good to me - appreciate the help ! yes i'm currently working on (6) so feel free to work on (1) and (2)

[shaneutt]

Sounds good, just please make sure to create sub-issues and place them on the project board, ty!