[Bug]: Bedrock Guardrail with during_call doesn't correctly use Guardrail output #15092
Description
What happened?
Define a Guardrail:
guardrails:
- guardrail_name: "guardrail-name"
litellm_params:
guardrail: bedrock
mode: "during_call"
guardrailIdentifier: "guardrail-name"
guardrailVersion: "DRAFT"
trace: "enabled"
aws_region_name: us-east-1
disable_exception_on_block: true
Make a request that should be blocked by that Guardrail. See that the request is not correctly blocked.
Guardrail result:
Output from LiteLLM:
Reviewing the code:
litellm/litellm/proxy/guardrails/guardrail_hooks/bedrock_guardrails.py
Lines 654 to 656 in f46f9d3
during_call hook uses _update_messages_with_updated_bedrock_guardrail_response which uses _apply_masking_to_messages to mask messsages. Compare that with post_call hook, which uses _apply_masking_to_response to replace the response (if applicable).
The docs suggest that PII masking requires pre_call, but the pre_call hook uses the same _update_messages_with_updated_bedrock_guardrail_response method as the during_call hook (which makes it seem like the during_call hook should be using something else).
My expectation for the during_call hook, based on the documentation, is:
- Request is sent to LLM and Guardrail at the same time
- Response is determined by whether the Guardrail intervened or not:
a) If the Guardrail responds with no intervention, the response is what we got back from the LLM
b) If the Guardrail responds that there was intervention, the response is what we got back from the Guardrail (if you've disabled exceptions)
Relevant log output
message=Bedrock AI response : {'action': 'GUARDRAIL_INTERVENED', 'actionReason': 'Guardrail blocked.', 'assessments': [{'invocationMetrics': {'guardrailCoverage': {'textCharacters': {'guarded': 3310, 'total': 3310}}, 'guardrailProcessingLatency': 799, 'usage': {'automatedReasoningPolicies': 0, 'automatedReasoningPolicyUnits': 0, 'contentPolicyImageUnits': 0, 'contentPolicyUnits': 4, 'contextualGroundingPolicyUnits': 0, 'sensitiveInformationPolicyFreeUnits': 0, 'sensitiveInformationPolicyUnits': 0, 'topicPolicyUnits': 4, 'wordPolicyUnits': 4}}, 'wordPolicy': {'customWords': [{'action': 'BLOCKED', 'detected': True, 'match': '[REDACTED]'}]}}], 'blockedResponse': 'Sorry, I cannot answer this question.', 'guardrailCoverage': {'textCharacters': {'guarded': 3310, 'total': 3310}}, 'output': [{'text': 'Sorry, I cannot answer this question.'}], 'outputs': [{'text': 'Sorry, I cannot answer this question.'}], 'usage': {'automatedReasoningPolicies': 0, 'automatedReasoningPolicyUnits': 0, 'contentPolicyImageUnits': 0, 'contentPolicyUnits': 4, 'contextualGroundingPolicyUnits': 0, 'sensitiveInformationPolicyFreeUnits': 0, 'sensitiveInformationPolicyUnits': 0, 'topicPolicyUnits': 4, 'wordPolicyUnits': 4}}
level=DEBUG
timestamp=2025-10-01T01:19:49.584748
message=Bedrock guardrail provided masked output, applying to messages
level=DEBUG
timestamp=2025-10-01T01:19:49.584934
message=litellm.acompletion(model=bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0) 200 OK
level=INFO
timestamp=2025-10-01T01:19:50.481648
message=Async Response: ModelResponse(id='chatcmpl-34d08bdc-84ed-4f32-9749-bc827a370983', created=1759281590, model='us.anthropic.claude-sonnet-4-20250514-v1:0', object='chat.completion', system_fingerprint=None, choices=[Choices(finish_reason='stop', index=0, message=Message(content='I understand you mentioned coffee, but I need to focus on gathering', role='assistant', tool_calls=None, function_call=None, provider_specific_fields=None))], usage=Usage(completion_tokens=58, prompt_tokens=1267, total_tokens=1325, completion_tokens_details=None, prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=None, cached_tokens=0, text_tokens=None, image_tokens=None, cache_creation_tokens=0), cache_creation_input_tokens=0, cache_read_input_tokens=0))
level=DEBUG
timestamp=2025-10-01T01:19:50.481912
Are you a ML Ops Team?
No
What LiteLLM version are you on ?
v1.77.5.rc.2
Twitter / LinkedIn details
No response