Bedrock integration #3417
Open
Bedrock integration #3417
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Added BedrockEnabled for toggling Bedrock mode - Added BedrockRegion for AWS region configuration - Added BedrockModel for model selection - Added BedrockContextWindow for context window size - Added BedrockThinkingEnabled for thinking mode toggle - Added BedrockTemperature for temperature control - Added BedrockSystemPromptActive for custom system prompts
- Added aws-sdk-bedrockruntime dependency - Created bedrock.rs module with BedrockApiClient - Client initializes with region and model from settings - Prepares for Converse API integration
- Updated aws-sdk-bedrockruntime from 1.51.0 to 1.112.0 - Updated Rust toolchain from 1.87.0 to 1.88.0 for compatibility
- Added converse_stream method with full message handling - Implemented tool configuration conversion from Q format to Bedrock - Added temperature and max_tokens configuration support - Handles thinking mode (auto-sets temperature to 1.0) - Converts chat history to Bedrock message format - Prepared for system prompt support
- Added bedrock_client field to ApiClient struct - Initialize Bedrock client when BedrockEnabled setting is true - Route send_message to Bedrock when enabled - Added Bedrock variant to SendMessageOutput enum - Bedrock takes priority over Q Developer when enabled - Added Clone and Debug derives to BedrockApiClient
- Convert Bedrock ConverseStreamOutput events to ChatResponseStream - Handle ContentBlockDelta for text and tool use - Handle ContentBlockStart for tool use events - Handle MessageStop for stream completion - Map Bedrock streaming events to Q CLI format - Proper error handling for stream errors
- Check BedrockEnabled setting before requiring auth - Skip login prompt when using Bedrock backend - Bedrock uses AWS credentials directly, not Q Developer auth - Allows seamless use of Bedrock without Q login
- Added list_foundation_models method to BedrockApiClient - Query Bedrock for available Claude models dynamically - Modified get_available_models to check for Bedrock mode - Added Bedrock-specific fallback models - Includes Claude 3.5 Sonnet v2, 3.5 Sonnet, and 3 Sonnet - Added aws-sdk-bedrock dependency for model listing API
- Check BedrockEnabled setting in fallback_model_id - Return Bedrock model from settings when in Bedrock mode - Ensures correct model is used for conversation initialization Note: Model display name still needs fixing for inference profiles
- Added full AssistantResponseMessage conversion with tool uses - Added UserInputMessage conversion with tool results - Convert tool result content blocks to Bedrock format - Handle both text and JSON tool results - Map tool result status (Success/Error) Note: Still encountering service error when sending follow-up with tool results. May need to investigate message format.
- Filter out empty messages from history - Ensure messages alternate between user/assistant roles - Skip text content in user messages that have tool results - Add checks to prevent sending empty content blocks Known issue: Tool result round-trip still encountering Bedrock service error. Tool execution works, but sending results back to Bedrock for final response needs further investigation.
Added detailed logging to /tmp/bedrock_api_calls.json Key findings: - ValidationException from Bedrock when sending tool results - Empty text block being sent in user message with tool results - Tool results not appearing in the logged messages - Issue: Tool results from history not being properly converted The problem is that tool results aren't making it into the Bedrock API call. Need to investigate how tool results flow through the conversation history.
The issue was duplicate user message addition. We were: 1. Converting the current message with tool results 2. Then adding ANOTHER user message with just text Fixed by: - Using as_ref() to avoid consuming context - Cloning tool result data instead of moving - Removing duplicate user message addition - Properly converting current message through convert_chat_message_to_bedrock Tool round-trip now works end-to-end: - Tool is called - Results are captured - Results sent back to Bedrock - Final response generated Tested successfully with S3 bucket listing!
- Add BedrockMaxTokens setting (separate from context window) - Implement inference profile auto-resolution via list-inference-profiles API - Add interactive model selection: q config bedrock-model - Filter models to show only ACTIVE text generation models - Add q config command with subcommands for all Bedrock settings - Add raw key methods to Settings for custom system prompts - Remove debug output and unused code - Fix all compiler warnings - Verify multi-region support (commercial, GovCloud, ISO, ISO-B) All features tested and working in production.
mcryan77
force-pushed
the
bedrock-integration
branch
from
37b7c1d to
dc73289
Compare
…ommand" This reverts commit dc73289.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
Add Amazon Bedrock Backend Support
Summary
This PR adds Amazon Bedrock as an alternative backend for Amazon Q Developer CLI, enabling users to interact with Claude and other foundation models directly through their AWS accounts without requiring Q Developer authentication.
Key Features
Core Integration
• Bedrock API Client: Full implementation using AWS Converse API with streaming support
• Automatic Backend Routing: Seamlessly switches between Q Developer and Bedrock based on configuration
• Tool Execution: All existing tools work with Bedrock (fs_read, fs_write, use_aws, execute_bash, etc.)
• Authentication Bypass: Uses AWS credentials directly when Bedrock mode is enabled
Configuration Commands
• q config bedrock <true|false> - Toggle Bedrock mode
• q config region - Set AWS region
• q config bedrock-model - Interactive model selection from available models
• q config max-tokens - Control output length (up to 200K)
• q config temperature <0.0-1.0> - Adjust response creativity
• q config thinking <true|false> - Enable extended reasoning mode
• q config system-prompt <add|enable|default|delete|list> - Manage custom system prompts
Advanced Features
• Custom System Prompts: Create, enable, and manage task-specific prompts
• Dynamic Model Discovery: Automatically queries and filters available text generation models
• Inference Profile Resolution: Auto-resolves models to cross-region inference profiles for better availability
• Temperature Validation: Enforces 0.0-1.0 range and blocks when thinking mode is active
• Settings Persistence: All configurations saved and preserved across sessions
Technical Implementation
New Files
• crates/q_cli/src/config.rs - Configuration command implementation (~600 lines)
Modified Files
• crates/q_cli/src/main.rs - Added config subcommand integration
• crates/q_cli/src/bedrock.rs - Bedrock client with tool support and streaming
• crates/q_cli/src/settings.rs - Added raw key methods for dynamic prompt storage
• README.md - Updated with Bedrock quick start and documentation links
Key Technical Details
• Separated max_tokens (output) from context window (input) settings
• Model filtering shows only ACTIVE text generation models
• Inference profile auto-resolution via list-inference-profiles API
• System prompts stored as bedrock.systemPrompt.{name} with active tracking
• Zero compiler warnings, clean release build
Testing
• ✅ All configuration commands tested and verified
• ✅ End-to-end conversations with Claude 4 Sonnet
• ✅ Tool execution validated (S3 operations, file operations)
• ✅ Settings persistence across sessions
• ✅ Error handling and validation
• ✅ Model selection and inference profile resolution
Usage Example
bash
Enable Bedrock
q config bedrock true
q config region us-west-2
Configure for code review
q config temperature 0.5
q config system-prompt add "reviewer" "You are an expert code reviewer."
q config system-prompt enable reviewer
Start chatting
q chat
Benefits
• No Q Developer Login Required: Direct AWS credential usage
• Cost Transparency: Bedrock usage bills directly to AWS account
• Model Flexibility: Access to all Bedrock text generation models
• Customization: System prompts and temperature control
• Backward Compatible: Existing Q Developer mode unchanged
Documentation
BEDROCK_USAGE_GUIDE.md
Prerequisites
• AWS credentials configured
• Amazon Bedrock access with model permissions
• IAM permissions: bedrock:InvokeModel, bedrock:ListFoundationModels, bedrock:ListInferenceProfiles
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.