diff --git a/BEDROCK_USAGE_GUIDE.md b/BEDROCK_USAGE_GUIDE.md new file mode 100644 index 0000000000..8730e90364 --- /dev/null +++ b/BEDROCK_USAGE_GUIDE.md @@ -0,0 +1,450 @@ +# Amazon Q Developer CLI - Bedrock Integration Usage Guide + +## Overview + +This guide covers how to use Amazon Q Developer CLI with Amazon Bedrock as the backend. The Bedrock integration allows you to use Claude models directly through your AWS account while maintaining all existing Q Developer CLI functionality and tools. + +## Prerequisites + +- AWS credentials configured (via `~/.aws/credentials` or environment variables) +- Access to Amazon Bedrock with Claude models enabled in your AWS account +- Amazon Q Developer CLI installed + +## Quick Start + +### 1. Enable Bedrock Mode + +```bash +q config bedrock true +``` + +### 2. Set Your AWS Region + +```bash +q config region us-west-2 +``` + +### 3. Configure Your Model + +List available models in your account: +```bash +q chat +/model +``` + +Set a specific model using settings: +```bash +q settings bedrock.model "us.anthropic.claude-sonnet-4-20250514-v1:0" +``` + +### 4. Start Chatting + +```bash +q chat +``` + +No login required when Bedrock mode is enabled! + +## Common Commands Quick Reference + +```bash +# Configuration +q config bedrock true # Enable Bedrock +q config region us-west-2 # Set region +q config max-tokens 8192 # Set max output tokens +q config temperature 0.7 # Set temperature +q config thinking true # Enable thinking mode + +# System Prompts +q config system-prompt add "name" "prompt text" +q config system-prompt enable "name" +q config system-prompt default +q config system-prompt list + +# Model Selection +q config bedrock-model # Interactive model picker + +# Settings +q settings list # View all settings +q settings bedrock.model "model-id" # Set model directly + +# Chat +q chat # Start chat +/model # Select model (in chat) +``` + +## Configuration Commands + +### Bedrock Mode Toggle + +Enable Bedrock backend: +```bash +q config bedrock true +``` + +Disable Bedrock backend (returns to Q Developer): +```bash +q config bedrock false +``` + +### Region Configuration + +Set AWS region for Bedrock API calls: +```bash +q config region us-east-1 +q config region us-west-2 +q config region eu-west-1 +q config region us-gov-west-1 +q config region us-iso-west-1 +q config region us-isob-east-1 (when available) +``` + +Supports all AWS commercial regions, GovCloud, ISO, and ISO-B regions. + +### Model Selection + +**Option 1: Interactive command-line selection (recommended)** +```bash +q config bedrock-model +``` +This queries your AWS account for available models and lets you select one interactively. + +**Option 2: Interactive in-chat selection** +```bash +q chat +/model +``` +This queries your AWS account for available Claude models and lets you select one during a chat session. + +**Option 3: Direct configuration** +```bash +q settings bedrock.model "anthropic.claude-3-5-sonnet-20241022-v2:0" +``` + +**Common Model IDs:** +- Claude 4 Sonnet (inference profile): `us.anthropic.claude-sonnet-4-20250514-v1:0` +- Claude 3.5 Sonnet v2: `anthropic.claude-3-5-sonnet-20241022-v2:0` +- Claude 3.5 Sonnet: `anthropic.claude-3-5-sonnet-20240620-v1:0` +- Claude 3 Opus: `anthropic.claude-3-opus-20240229-v1:0` + +### Max Output Tokens + +Set the maximum number of tokens in the model's response (up to 200,000): +```bash +q config max-tokens 4096 +q config max-tokens 8192 +q config max-tokens 16384 +``` + +Default: 4096 tokens + +**Note:** This controls the output length, not the input context window. Different models have different maximum token limits. + +### Thinking Mode + +Enable extended thinking mode (automatically sets temperature to 1.0): +```bash +q config thinking true +``` + +Disable thinking mode: +```bash +q config thinking false +``` + +When thinking mode is enabled, the model uses extended reasoning and temperature is locked at 1.0. + +### Temperature Control + +Set temperature (0.0 to 1.0): +```bash +q config temperature 0.7 +q config temperature 0.3 +q config temperature 1.0 +``` + +**Note:** Temperature can only be configured when thinking mode is disabled. If thinking mode is enabled, temperature is automatically set to 1.0. + +### Custom System Prompts + +**Add a new system prompt:** +```bash +q config system-prompt add "python-expert" "You are a Python expert. Focus on Pythonic code and best practices." +q config system-prompt add "security" "You are a security expert focused on identifying vulnerabilities." +q config system-prompt add "concise" "Be extremely brief. One sentence maximum." +``` + +**List all system prompts:** +```bash +q config system-prompt list +``` + +Output shows all prompts with an `(active)` marker for the currently enabled prompt: +``` +Custom system prompts: + - python-expert (active) + You are a Python expert. Focus on Pythonic code and best pra... + - security + You are a security expert focused on identifying vulnerabili... + - concise + Be extremely brief. One sentence maximum. +``` + +**Enable a system prompt:** +```bash +q config system-prompt enable python-expert +``` + +**Default system prompt:** +```bash +q config system-prompt disable +``` + +**Delete a system prompt:** +```bash +q config system-prompt delete concise +``` + +If you delete the active prompt, it will be automatically deactivated. + +**Default behavior:** When no custom system prompt is active, the model uses its default system prompt. + +## Settings Management + +### View All Bedrock Settings + +```bash +q settings list --all | grep bedrock +``` + +### View Configured Settings + +```bash +q settings list +``` + +### Get a Specific Setting + +```bash +q settings bedrock.enabled +q settings bedrock.region +q settings bedrock.model +``` + +### Set a Setting Directly + +```bash +q settings bedrock.enabled true +q settings bedrock.region "us-west-2" +q settings bedrock.model "anthropic.claude-3-5-sonnet-20241022-v2:0" +``` + +### Delete a Setting + +```bash +q settings --delete bedrock.temperature +``` + +## Complete Configuration Example + +Here's a complete setup workflow: + +```bash +# 1. Enable Bedrock mode +q config bedrock true + +# 2. Set your region +q config region us-west-2 + +# 3. Set context window +q config context-window 200000 + +# 4. Configure temperature +q config temperature 0.7 + +# 5. Add custom system prompts +q config system-prompt add "code-reviewer" "You are an expert code reviewer. Focus on best practices, security, and performance." +q config system-prompt enable code-reviewer + +# 6. Start chatting +q chat + +# 7. Inside chat, select your model +/model +``` + +## Tool Support + +All existing Q Developer CLI tools work seamlessly with Bedrock: + +- **File operations:** `fs_read`, `fs_write` +- **AWS operations:** `use_aws` (S3, EC2, Lambda, etc.) +- **Bash execution:** `execute_bash` +- **And all other tools** + + +## Authentication + +**No login required!** When Bedrock mode is enabled, authentication is bypassed. The CLI uses your AWS credentials directly. + +To check your configuration: +```bash +q settings list +``` + +## Switching Between Q Developer and Bedrock + +**Switch to Bedrock:** +```bash +q config bedrock true +``` + +**Switch back to Q Developer:** +```bash +q config bedrock false +q login # Re-authenticate with Q Developer +``` + +All your Bedrock settings are preserved when you switch back and forth. + +## Supported Models + +The `/model` command and `q config bedrock-model` dynamically query your AWS Bedrock account and display all available **text generation models**. This includes: + +- **Anthropic:** Claude 4, Claude 3.5, Claude 3 (Opus, Sonnet, Haiku) +- **Amazon:** Titan Text models +- **Meta:** Llama 2, Llama 3 models +- **Mistral AI:** Mistral and Mixtral models +- **Cohere:** Command models +- **AI21 Labs:** Jurassic models + +### Model Filtering + +The CLI automatically filters models to show only those suitable for text generation: + +- ✅ Shows only text generation models +- ✅ Shows only ACTIVE models (excludes unreleased or deprecated models) + +### Inference Profiles + +Models with `us.` prefix (e.g., `us.anthropic.claude-sonnet-4-20250514-v1:0`) are **inference profiles** that provide: + +- Better availability through cross-region routing +- Automatic failover if a region is unavailable +- Recommended for production use + +The CLI automatically resolves models to inference profiles when available. You can use either the direct model ID or the inference profile ID. + +**Usage:** +```bash +q config bedrock-model # Interactive selection with all available models +q chat +/model # In-chat selection +``` + + +## Advanced Usage + +### Using Inference Profiles + +For Claude 4 models, use inference profiles for better availability: +```bash +q settings bedrock.model "us.anthropic.claude-sonnet-4-20250514-v1:0" +``` + +### Multiple System Prompts for Different Tasks + +Create task-specific prompts: +```bash +q config system-prompt add "debugging" "You are a debugging expert. Focus on root cause analysis." +q config system-prompt add "documentation" "You are a technical writer. Create clear, concise documentation." +q config system-prompt add "architecture" "You are a software architect. Focus on system design and scalability." +``` + +Switch between them as needed: +```bash +q config system-prompt enable debugging +# Work on debugging... + +q config system-prompt enable documentation +# Write documentation... +``` + +### Temperature Strategies + +- **Creative tasks:** `q config temperature 0.9` +- **Balanced:** `q config temperature 0.7` +- **Deterministic/factual:** `q config temperature 0.3` +- **Extended thinking:** `q config thinking true` (locks to 1.0) + +### Large Context Windows + +For working with large codebases: +```bash +q config context-window 200000 +``` + +This allows the model to see more of your code at once. + +## Configuration Reference + +| Setting | Command | Values | Default | +|---------|---------|--------|---------| +| Bedrock Mode | `q config bedrock ` | `true`, `false` | `false` | +| Region | `q config region ` | Any AWS region | `us-east-1` | +| Model | `q settings bedrock.model ` | Model ID string | None | +| Context Window | `q config context-window ` | Number (e.g., 8192) | 4096 | +| Thinking Mode | `q config thinking ` | `true`, `false` | `false` | +| Temperature | `q config temperature ` | 0.0 to 1.0 | 0.7 | +| Active Prompt | `q config system-prompt enable ` | Prompt name | None | + +## Tips + +1. **Start simple** - Enable Bedrock, set region, start chatting +2. **Use inference profiles** - Models are automatically resolved to inference profiles when available +3. **Create system prompts** - Build a library for different tasks, switch with `enable` or return to default +4. **Adjust temperature** - Lower (0.3) for factual, higher (0.9) for creative +5. **Use interactive model selection** - Run `q config bedrock-model` to browse and select models +6. **Monitor costs** - Bedrock usage bills to your AWS account + +## Best Practices + +1. **Start with defaults:** Enable Bedrock mode and set your region, then adjust other settings as needed +2. **Use inference profiles:** For Claude 4 models, use inference profiles (us.* prefix) for better availability +3. **Match context window to task:** Use larger windows for complex codebases, smaller for focused tasks +4. **Create reusable prompts:** Build a library of system prompts for different types of work +5. **Test temperature settings:** Different tasks benefit from different temperature values +6. **Monitor costs:** Bedrock usage is billed to your AWS account - monitor in AWS Cost Explorer + +## Getting Help + +View all config commands: +```bash +q config --help +``` + +View system-prompt commands: +```bash +q config system-prompt --help +``` + +View settings commands: +```bash +q settings --help +``` + +## Learn More + +- **AWS Bedrock:** https://aws.amazon.com/bedrock/ +- **Claude Models:** https://docs.anthropic.com/claude/docs + +## Summary + +The Bedrock integration provides: +- ✅ All existing Q Developer CLI tools and features +- ✅ No separate authentication required +- ✅ Flexible configuration options +- ✅ Custom system prompts +- ✅ Temperature and context window control +- ✅ Extended thinking mode support + diff --git a/Cargo.lock b/Cargo.lock index 5f311f4e38..f2e2a365cc 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -245,7 +245,7 @@ dependencies = [ "aws-http", "aws-runtime", "aws-smithy-async", - "aws-smithy-http 0.62.4", + "aws-smithy-http 0.62.5", "aws-smithy-json", "aws-smithy-runtime", "aws-smithy-runtime-api", @@ -488,7 +488,7 @@ dependencies = [ "aws-sdk-ssooidc", "aws-sdk-sts", "aws-smithy-async", - "aws-smithy-http 0.62.4", + "aws-smithy-http 0.62.5", "aws-smithy-json", "aws-smithy-runtime", "aws-smithy-runtime-api", @@ -552,14 +552,15 @@ dependencies = [ [[package]] name = "aws-runtime" -version = "1.5.12" +version = "1.5.13" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bfa006bb32360ed90ac51203feafb9d02e3d21046e1fd3a450a404b90ea73e5d" +checksum = "9f2402da1a5e16868ba98725e5d73f26b8116eaa892e56f2cd0bf5eec7985f70" dependencies = [ "aws-credential-types", "aws-sigv4", "aws-smithy-async", - "aws-smithy-http 0.62.4", + "aws-smithy-eventstream", + "aws-smithy-http 0.62.5", "aws-smithy-runtime", "aws-smithy-runtime-api", "aws-smithy-types", @@ -574,6 +575,53 @@ dependencies = [ "uuid", ] +[[package]] +name = "aws-sdk-bedrock" +version = "1.119.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "18303897ff30d21f7da47d5921ee1ea68d2a2a7c12544a916cece9cacc502d38" +dependencies = [ + "aws-credential-types", + "aws-runtime", + "aws-smithy-async", + "aws-smithy-http 0.62.5", + "aws-smithy-json", + "aws-smithy-runtime", + "aws-smithy-runtime-api", + "aws-smithy-types", + "aws-types", + "bytes", + "fastrand", + "http 0.2.12", + "regex-lite", + "tracing", +] + +[[package]] +name = "aws-sdk-bedrockruntime" +version = "1.112.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c06c037e6823696d752702ec2bad758d3cf95d1b92b712c8ac7e93824b5e2391" +dependencies = [ + "aws-credential-types", + "aws-runtime", + "aws-sigv4", + "aws-smithy-async", + "aws-smithy-eventstream", + "aws-smithy-http 0.62.5", + "aws-smithy-json", + "aws-smithy-runtime", + "aws-smithy-runtime-api", + "aws-smithy-types", + "aws-types", + "bytes", + "fastrand", + "http 0.2.12", + "hyper 0.14.32", + "regex-lite", + "tracing", +] + [[package]] name = "aws-sdk-cognitoidentity" version = "1.87.0" @@ -583,7 +631,7 @@ dependencies = [ "aws-credential-types", "aws-runtime", "aws-smithy-async", - "aws-smithy-http 0.62.4", + "aws-smithy-http 0.62.5", "aws-smithy-json", "aws-smithy-runtime", "aws-smithy-runtime-api", @@ -605,7 +653,7 @@ dependencies = [ "aws-credential-types", "aws-runtime", "aws-smithy-async", - "aws-smithy-http 0.62.4", + "aws-smithy-http 0.62.5", "aws-smithy-json", "aws-smithy-runtime", "aws-smithy-runtime-api", @@ -627,7 +675,7 @@ dependencies = [ "aws-credential-types", "aws-runtime", "aws-smithy-async", - "aws-smithy-http 0.62.4", + "aws-smithy-http 0.62.5", "aws-smithy-json", "aws-smithy-runtime", "aws-smithy-runtime-api", @@ -649,7 +697,7 @@ dependencies = [ "aws-credential-types", "aws-runtime", "aws-smithy-async", - "aws-smithy-http 0.62.4", + "aws-smithy-http 0.62.5", "aws-smithy-json", "aws-smithy-query", "aws-smithy-runtime", @@ -665,12 +713,13 @@ dependencies = [ [[package]] name = "aws-sigv4" -version = "1.3.5" +version = "1.3.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bffc03068fbb9c8dd5ce1c6fb240678a5cffb86fb2b7b1985c999c4b83c8df68" +checksum = "c35452ec3f001e1f2f6db107b6373f1f48f05ec63ba2c5c9fa91f07dad32af11" dependencies = [ "aws-credential-types", - "aws-smithy-http 0.62.4", + "aws-smithy-eventstream", + "aws-smithy-http 0.62.5", "aws-smithy-runtime-api", "aws-smithy-types", "bytes", @@ -698,9 +747,9 @@ dependencies = [ [[package]] name = "aws-smithy-eventstream" -version = "0.60.12" +version = "0.60.13" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9656b85088f8d9dc7ad40f9a6c7228e1e8447cdf4b046c87e152e0805dea02fa" +checksum = "e29a304f8319781a39808847efb39561351b1bb76e933da7aa90232673638658" dependencies = [ "aws-smithy-types", "bytes", @@ -730,15 +779,17 @@ dependencies = [ [[package]] name = "aws-smithy-http" -version = "0.62.4" +version = "0.62.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3feafd437c763db26aa04e0cc7591185d0961e64c61885bece0fb9d50ceac671" +checksum = "445d5d720c99eed0b4aa674ed00d835d9b1427dd73e04adaf2f94c6b2d6f9fca" dependencies = [ + "aws-smithy-eventstream", "aws-smithy-runtime-api", "aws-smithy-types", "bytes", "bytes-utils", "futures-core", + "futures-util", "http 0.2.12", "http 1.3.1", "http-body 0.4.6", @@ -750,9 +801,9 @@ dependencies = [ [[package]] name = "aws-smithy-http-client" -version = "1.1.3" +version = "1.1.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1053b5e587e6fa40ce5a79ea27957b04ba660baa02b28b7436f64850152234f1" +checksum = "623254723e8dfd535f566ee7b2381645f8981da086b5c4aa26c0c41582bb1d2c" dependencies = [ "aws-smithy-async", "aws-smithy-protocol-test", @@ -786,9 +837,9 @@ dependencies = [ [[package]] name = "aws-smithy-json" -version = "0.61.6" +version = "0.61.7" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "cff418fc8ec5cadf8173b10125f05c2e7e1d46771406187b2c878557d4503390" +checksum = "2db31f727935fc63c6eeae8b37b438847639ec330a9161ece694efba257e0c54" dependencies = [ "aws-smithy-types", ] @@ -804,9 +855,9 @@ dependencies = [ [[package]] name = "aws-smithy-protocol-test" -version = "0.63.5" +version = "0.63.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "09e4a766a447bf2aca69100278a6777cffcef2f97199f2443d481c698dd2887c" +checksum = "fa808d23a8edf0da73f6812d06d8c0a48d70f05d2d3696362982aad11ee475b7" dependencies = [ "assert-json-diff", "aws-smithy-runtime-api", @@ -833,12 +884,12 @@ dependencies = [ [[package]] name = "aws-smithy-runtime" -version = "1.9.3" +version = "1.9.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "40ab99739082da5347660c556689256438defae3bcefd66c52b095905730e404" +checksum = "0bbe9d018d646b96c7be063dd07987849862b0e6d07c778aad7d93d1be6c1ef0" dependencies = [ "aws-smithy-async", - "aws-smithy-http 0.62.4", + "aws-smithy-http 0.62.5", "aws-smithy-http-client", "aws-smithy-observability", "aws-smithy-runtime-api", @@ -858,9 +909,9 @@ dependencies = [ [[package]] name = "aws-smithy-runtime-api" -version = "1.9.1" +version = "1.9.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "3683c5b152d2ad753607179ed71988e8cfd52964443b4f74fd8e552d0bbfeb46" +checksum = "ec7204f9fd94749a7c53b26da1b961b4ac36bf070ef1e0b94bb09f79d4f6c193" dependencies = [ "aws-smithy-async", "aws-smithy-types", @@ -875,9 +926,9 @@ dependencies = [ [[package]] name = "aws-smithy-types" -version = "1.3.3" +version = "1.3.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9f5b3a7486f6690ba25952cabf1e7d75e34d69eaff5081904a47bc79074d6457" +checksum = "25f535879a207fce0db74b679cfc3e91a3159c8144d717d55f5832aea9eef46e" dependencies = [ "base64-simd", "bytes", @@ -923,9 +974,9 @@ dependencies = [ [[package]] name = "aws-types" -version = "1.3.9" +version = "1.3.10" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e2fd329bf0e901ff3f60425691410c69094dc2a1f34b331f37bfc4e9ac1565a1" +checksum = "d79fb68e3d7fe5d4833ea34dc87d2e97d26d3086cb3da660bb6b1f76d98680b6" dependencies = [ "aws-credential-types", "aws-smithy-async", @@ -1460,6 +1511,8 @@ dependencies = [ "aws-config", "aws-credential-types", "aws-runtime", + "aws-sdk-bedrock", + "aws-sdk-bedrockruntime", "aws-sdk-cognitoidentity", "aws-sdk-ssooidc", "aws-smithy-async", diff --git a/Cargo.toml b/Cargo.toml index 3eced0d620..5dece75822 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -25,6 +25,8 @@ async-trait = "0.1.87" aws-config = "1.0.3" aws-credential-types = "1.0.3" aws-runtime = "1.4.4" +aws-sdk-bedrock = "1.119.0" +aws-sdk-bedrockruntime = "1.112.0" aws-sdk-cognitoidentity = "1.51.0" aws-sdk-ssooidc = "1.51.0" aws-smithy-async = "1.2.2" diff --git a/crates/chat-cli/Cargo.toml b/crates/chat-cli/Cargo.toml index 68370bc3d0..8adc3ff893 100644 --- a/crates/chat-cli/Cargo.toml +++ b/crates/chat-cli/Cargo.toml @@ -28,6 +28,8 @@ async-trait.workspace = true aws-config.workspace = true aws-credential-types.workspace = true aws-runtime.workspace = true +aws-sdk-bedrock.workspace = true +aws-sdk-bedrockruntime.workspace = true aws-sdk-cognitoidentity.workspace = true aws-sdk-ssooidc.workspace = true aws-smithy-async.workspace = true diff --git a/crates/chat-cli/src/api_client/bedrock.rs b/crates/chat-cli/src/api_client/bedrock.rs new file mode 100644 index 0000000000..de44782227 --- /dev/null +++ b/crates/chat-cli/src/api_client/bedrock.rs @@ -0,0 +1,503 @@ +use aws_config::BehaviorVersion; +use aws_sdk_bedrockruntime::Client as BedrockClient; +use aws_sdk_bedrockruntime::operation::converse_stream::ConverseStreamOutput; +use aws_sdk_bedrockruntime::types::{ + ContentBlock, ConversationRole, Message, SystemContentBlock, Tool, ToolConfiguration, + ToolInputSchema, ToolSpecification, +}; +use eyre::Result; +use std::io::Write; + +use crate::api_client::model::{ConversationState, UserInputMessageContext}; +use crate::database::settings::Setting; +use crate::database::Database; + +#[derive(Clone, Debug)] +pub struct BedrockApiClient { + client: BedrockClient, + model_id: String, + database: Database, +} + +impl BedrockApiClient { + pub async fn new(database: Database) -> Result { + let region = database + .settings + .get(Setting::BedrockRegion) + .and_then(|v| v.as_str()) + .unwrap_or("us-east-1"); + + // AWS SDK automatically resolves the correct endpoint based on region partition: + // - Commercial: bedrock-runtime.{region}.amazonaws.com + // - GovCloud: bedrock-runtime.{region}.amazonaws-us-gov.com + // - ISO: bedrock-runtime.{region}.c2s.ic.gov + // - ISO-B: bedrock-runtime.{region}.sc2s.sgov.gov + let config = aws_config::defaults(BehaviorVersion::latest()) + .region(aws_config::Region::new(region.to_string())) + .load() + .await; + + let client = BedrockClient::new(&config); + + let model_id = database + .settings + .get(Setting::BedrockModel) + .and_then(|v| v.as_str()) + .unwrap_or("anthropic.claude-3-sonnet-20240229-v1:0") + .to_string(); + + Ok(Self { + client, + model_id, + database, + }) + } + + pub async fn converse_stream( + &self, + conversation: ConversationState, + ) -> Result { + let ConversationState { + conversation_id: _, + user_input_message, + history, + } = conversation; + + // Use model from user_input_message if available, otherwise use the one from settings + let model_id = user_input_message + .model_id + .as_deref() + .unwrap_or(&self.model_id); + + // Resolve to inference profile if available + let resolved_model_id = self.resolve_model_id(model_id).await?; + + // Build messages from history and current message + let mut messages = Vec::new(); + + // Add history messages + if let Some(hist) = history { + for msg in hist { + let converted = self.convert_chat_message_to_bedrock(msg)?; + // Only add non-empty messages + if !converted.is_empty() { + messages.extend(converted); + } + } + } + + // Ensure we have alternating user/assistant messages + // Bedrock requires strict alternation + let mut filtered_messages = Vec::new(); + let mut last_role: Option = None; + + for msg in messages { + let current_role = msg.role().clone(); + if Some(¤t_role) != last_role.as_ref() { + filtered_messages.push(msg); + last_role = Some(current_role); + } + } + + messages = filtered_messages; + + // Add current user message + let converted_current = self.convert_chat_message_to_bedrock( + crate::api_client::model::ChatMessage::UserInputMessage(user_input_message.clone()) + )?; + + messages.extend(converted_current); + + tracing::debug!("Sending {} messages to Bedrock", messages.len()); + for (i, msg) in messages.iter().enumerate() { + tracing::debug!("Message {}: role={:?}, content_blocks={}", i, msg.role(), msg.content().len()); + } + + // Build tool configuration if tools are present + let tool_config = user_input_message + .user_input_message_context + .and_then(|ctx| self.build_tool_configuration(ctx)); + + // Get inference parameters + let temperature = self.get_temperature(); + let max_tokens = self.get_max_tokens(); + + // Log the full request to a file for debugging + let debug_file = std::path::Path::new("/tmp/bedrock_api_calls.json"); + if let Ok(mut file) = std::fs::OpenOptions::new() + .create(true) + .append(true) + .open(debug_file) + { + let debug_data = serde_json::json!({ + "timestamp": format!("{:?}", std::time::SystemTime::now()), + "model_id": &resolved_model_id, + "message_count": messages.len(), + "messages": messages.iter().map(|m| { + serde_json::json!({ + "role": format!("{:?}", m.role()), + "content_blocks": m.content().iter().map(|c| { + match c { + ContentBlock::Text(t) => serde_json::json!({"type": "text", "text": t}), + ContentBlock::ToolUse(tu) => serde_json::json!({ + "type": "toolUse", + "toolUseId": tu.tool_use_id(), + "name": tu.name(), + "input": format!("{:?}", tu.input()) + }), + ContentBlock::ToolResult(tr) => serde_json::json!({ + "type": "toolResult", + "toolUseId": tr.tool_use_id(), + "status": format!("{:?}", tr.status()), + "content_len": tr.content().len() + }), + _ => serde_json::json!({"type": "unknown"}) + } + }).collect::>() + }) + }).collect::>(), + "tool_config_present": tool_config.is_some(), + "temperature": temperature, + "max_tokens": max_tokens + }); + let _ = writeln!(file, "{}", serde_json::to_string_pretty(&debug_data).unwrap()); + } + + // Build the request + let mut request = self + .client + .converse_stream() + .model_id(&resolved_model_id) + .set_messages(Some(messages)); + + if let Some(tool_cfg) = tool_config { + request = request.tool_config(tool_cfg); + } + + // Set system prompt if configured + if let Some(system_prompt) = self.get_system_prompt() { + request = request.system( + SystemContentBlock::Text(system_prompt) + ); + } + + // Set inference config + request = request.inference_config( + aws_sdk_bedrockruntime::types::InferenceConfiguration::builder() + .temperature(temperature) + .max_tokens(max_tokens) + .build(), + ); + + let response = request.send().await?; + Ok(response) + } + + fn convert_chat_message_to_bedrock( + &self, + msg: crate::api_client::model::ChatMessage, + ) -> Result> { + use crate::api_client::model::ChatMessage; + + match msg { + ChatMessage::UserInputMessage(user_msg) => { + let mut content_blocks = vec![]; + + // Check if we have tool results + let has_tool_results = user_msg.user_input_message_context + .as_ref() + .and_then(|ctx| ctx.tool_results.as_ref()) + .map(|results| !results.is_empty()) + .unwrap_or(false); + + tracing::debug!("UserInputMessage: has_tool_results={}, content_len={}", + has_tool_results, user_msg.content.len()); + + // Only add text content if we don't have tool results AND text is not empty + // (Bedrock expects tool results in a separate user message without text) + if !has_tool_results && !user_msg.content.is_empty() { + content_blocks.push(ContentBlock::Text(user_msg.content.clone())); + } + + // Add tool results if present (use as_ref to avoid moving) + if let Some(ref ctx) = user_msg.user_input_message_context { + tracing::debug!("Has context, tool_results present: {}", ctx.tool_results.is_some()); + if let Some(ref tool_results) = ctx.tool_results { + tracing::debug!("Processing {} tool results", tool_results.len()); + for result in tool_results { + let tool_result_content: Vec<_> = result.content.iter().filter_map(|c| { + match c { + crate::api_client::model::ToolResultContentBlock::Json(doc) => { + // Convert JSON to text representation + Some(aws_sdk_bedrockruntime::types::ToolResultContentBlock::Text( + format!("{:?}", doc) + )) + } + crate::api_client::model::ToolResultContentBlock::Text(text) => { + Some(aws_sdk_bedrockruntime::types::ToolResultContentBlock::Text(text.clone())) + } + } + }).collect(); + + let status = match result.status { + crate::api_client::model::ToolResultStatus::Success => { + aws_sdk_bedrockruntime::types::ToolResultStatus::Success + } + crate::api_client::model::ToolResultStatus::Error => { + aws_sdk_bedrockruntime::types::ToolResultStatus::Error + } + }; + + content_blocks.push(ContentBlock::ToolResult( + aws_sdk_bedrockruntime::types::ToolResultBlock::builder() + .tool_use_id(result.tool_use_id.clone()) + .set_content(Some(tool_result_content)) + .status(status) + .build() + .map_err(|e| eyre::eyre!("Failed to build tool result: {}", e))? + )); + } + } + } + + // Don't send message if no content blocks + if content_blocks.is_empty() { + return Ok(vec![]); + } + + // Filter out any empty text blocks + content_blocks.retain(|block| { + if let ContentBlock::Text(t) = block { + !t.is_empty() + } else { + true + } + }); + + // Check again after filtering + if content_blocks.is_empty() { + return Ok(vec![]); + } + + Ok(vec![Message::builder() + .role(ConversationRole::User) + .set_content(Some(content_blocks)) + .build() + .map_err(|e| eyre::eyre!("Failed to build user message: {}", e))?]) + } + ChatMessage::AssistantResponseMessage(assistant_msg) => { + let mut content_blocks = vec![]; + + // Add text content + if !assistant_msg.content.is_empty() { + content_blocks.push(ContentBlock::Text(assistant_msg.content)); + } + + // Add tool uses + if let Some(tool_uses) = assistant_msg.tool_uses { + for tool_use in tool_uses { + content_blocks.push(ContentBlock::ToolUse( + aws_sdk_bedrockruntime::types::ToolUseBlock::builder() + .tool_use_id(tool_use.tool_use_id) + .name(tool_use.name) + .input(tool_use.input.into()) + .build() + .map_err(|e| eyre::eyre!("Failed to build tool use: {}", e))? + )); + } + } + + // Don't send message if no content blocks + if content_blocks.is_empty() { + return Ok(vec![]); + } + + Ok(vec![Message::builder() + .role(ConversationRole::Assistant) + .set_content(Some(content_blocks)) + .build() + .map_err(|e| eyre::eyre!("Failed to build assistant message: {}", e))?]) + } + } + } + + fn build_tool_configuration(&self, ctx: UserInputMessageContext) -> Option { + let tools = ctx.tools?; + + let tool_specs: Vec = tools + .into_iter() + .filter_map(|tool| { + match tool { + crate::api_client::model::Tool::ToolSpecification(spec) => { + let input_schema = if let Some(json_doc) = spec.input_schema.json { + ToolInputSchema::Json(json_doc.into()) + } else { + return None; + }; + + ToolSpecification::builder() + .name(spec.name) + .description(spec.description) + .input_schema(input_schema) + .build() + .ok() + } + } + }) + .collect(); + + if tool_specs.is_empty() { + return None; + } + + ToolConfiguration::builder() + .set_tools(Some(tool_specs.into_iter().map(Tool::ToolSpec).collect())) + .build() + .ok() + } + + fn get_temperature(&self) -> f32 { + // If thinking is enabled, temperature is always 1.0 + if self + .database + .settings + .get(Setting::BedrockThinkingEnabled) + .and_then(|v| v.as_bool()) + .unwrap_or(false) + { + return 1.0; + } + + // Otherwise use configured temperature or default + self.database + .settings + .get(Setting::BedrockTemperature) + .and_then(|v| v.as_f64()) + .map(|v| v as f32) + .unwrap_or(1.0) + } + + fn get_max_tokens(&self) -> i32 { + // Max tokens for OUTPUT generation + // Claude models support up to 200000 output tokens + self.database + .settings + .get(Setting::BedrockMaxTokens) + .and_then(|v| v.as_i64()) + .map(|v| v.min(200000) as i32) + .unwrap_or(4096) + } + + fn get_system_prompt(&self) -> Option { + // Check if a custom system prompt is active + if let Some(active_name) = self.database.settings.get(Setting::BedrockSystemPromptActive) + .and_then(|v| v.as_str()) + { + let key = format!("bedrock.systemPrompt.{}", active_name); + if let Some(prompt) = self.database.settings.get_raw(&key) + .and_then(|v| v.as_str()) + { + return Some(prompt.to_string()); + } + } + None + } + + pub async fn list_foundation_models(&self) -> Result> { + use aws_sdk_bedrockruntime::config::Region; + + // Create a Bedrock client (not runtime) for listing models + let bedrock_config = aws_config::defaults(BehaviorVersion::latest()) + .region(Region::new( + self.database + .settings + .get(Setting::BedrockRegion) + .and_then(|v| v.as_str()) + .unwrap_or("us-east-1") + .to_string(), + )) + .load() + .await; + + let bedrock_client = aws_sdk_bedrock::Client::new(&bedrock_config); + + let response = bedrock_client + .list_foundation_models() + .send() + .await?; + + let models: Vec = response + .model_summaries() + .iter() + .filter(|m| { + // Filter for text models that are ACTIVE or don't have lifecycle info + let is_text = m.output_modalities().contains(&aws_sdk_bedrock::types::ModelModality::Text); + let is_available = m.model_lifecycle() + .map(|lifecycle| matches!(lifecycle.status(), aws_sdk_bedrock::types::FoundationModelLifecycleStatus::Active)) + .unwrap_or(true); // If no lifecycle info, assume available + is_text && is_available + }) + .map(|m| m.model_id().to_string()) + .collect(); + + Ok(models) + } + + /// List available inference profiles + async fn list_inference_profiles(&self) -> Result> { + use aws_sdk_bedrockruntime::config::Region; + + let bedrock_config = aws_config::defaults(BehaviorVersion::latest()) + .region(Region::new( + self.database + .settings + .get(Setting::BedrockRegion) + .and_then(|v| v.as_str()) + .unwrap_or("us-east-1") + .to_string(), + )) + .load() + .await; + + let bedrock_client = aws_sdk_bedrock::Client::new(&bedrock_config); + + let response = bedrock_client + .list_inference_profiles() + .send() + .await?; + + let profiles: Vec = response + .inference_profile_summaries() + .iter() + .map(|p| p.inference_profile_id().to_string()) + .collect(); + + Ok(profiles) + } + + /// Resolves a model ID to its inference profile if available, otherwise returns the original ID + pub async fn resolve_model_id(&self, model_id: &str) -> Result { + // If already an inference profile, return as-is + if model_id.starts_with("us.") { + return Ok(model_id.to_string()); + } + + // Get all available inference profiles + let inference_profiles = self.list_inference_profiles().await?; + + // Try the simple approach: prepend "us." and see if it exists + let inference_profile_candidate = format!("us.{}", model_id); + if inference_profiles.contains(&inference_profile_candidate) { + return Ok(inference_profile_candidate); + } + + // If not found, try to find any inference profile that matches + let inference_profile = inference_profiles + .iter() + .find(|p| p.contains(model_id)) + .cloned(); + + // Return inference profile if found, otherwise return original + Ok(inference_profile.unwrap_or_else(|| model_id.to_string())) + } +} diff --git a/crates/chat-cli/src/api_client/mod.rs b/crates/chat-cli/src/api_client/mod.rs index 898352b01f..e7397810b9 100644 --- a/crates/chat-cli/src/api_client/mod.rs +++ b/crates/chat-cli/src/api_client/mod.rs @@ -8,6 +8,7 @@ mod opt_out; pub mod profile; mod retry_classifier; pub mod send_message_output; +pub mod bedrock; use std::sync::Arc; use std::time::Duration; @@ -37,6 +38,7 @@ pub use error::ApiClientError; use error::{ ConverseStreamError, ConverseStreamErrorKind, + ConverseStreamSdkError, }; use parking_lot::Mutex; pub use profile::list_available_profiles; @@ -99,6 +101,7 @@ pub struct ApiClient { client: CodewhispererClient, streaming_client: Option, sigv4_streaming_client: Option, + bedrock_client: Option, mock_client: Option>>>>, profile: Option, model_cache: ModelCache, @@ -139,6 +142,7 @@ impl ApiClient { client, streaming_client: None, sigv4_streaming_client: None, + bedrock_client: None, mock_client: None, profile: None, model_cache: Arc::new(RwLock::new(None)), @@ -214,10 +218,29 @@ impl ApiClient { None }; + // Initialize Bedrock client if enabled + let bedrock_client = if database + .settings + .get(Setting::BedrockEnabled) + .and_then(|v| v.as_bool()) + .unwrap_or(false) + { + match bedrock::BedrockApiClient::new(database.clone()).await { + Ok(client) => Some(client), + Err(err) => { + error!("Failed to initialize Bedrock client: {err}"); + None + } + } + } else { + None + }; + Ok(Self { client, streaming_client, sigv4_streaming_client, + bedrock_client, mock_client: None, profile, model_cache: Arc::new(RwLock::new(None)), @@ -387,6 +410,22 @@ impl ApiClient { ) -> Result { debug!("Sending conversation: {:#?}", conversation); + // Route to Bedrock if enabled + if let Some(bedrock_client) = &self.bedrock_client { + match bedrock_client.converse_stream(conversation).await { + Ok(response) => return Ok(SendMessageOutput::Bedrock(response)), + Err(err) => { + error!("Bedrock API error: {err}"); + return Err(ConverseStreamError::new( + ConverseStreamErrorKind::Unknown { + reason_code: err.to_string(), + }, + None::, + )); + } + } + } + let ConversationState { conversation_id, user_input_message, diff --git a/crates/chat-cli/src/api_client/send_message_output.rs b/crates/chat-cli/src/api_client/send_message_output.rs index 43c15ab660..18d3c61a84 100644 --- a/crates/chat-cli/src/api_client/send_message_output.rs +++ b/crates/chat-cli/src/api_client/send_message_output.rs @@ -9,6 +9,7 @@ pub enum SendMessageOutput { amzn_codewhisperer_streaming_client::operation::generate_assistant_response::GenerateAssistantResponseOutput, ), QDeveloper(amzn_qdeveloper_streaming_client::operation::send_message::SendMessageOutput), + Bedrock(aws_sdk_bedrockruntime::operation::converse_stream::ConverseStreamOutput), Mock(Vec), } @@ -17,6 +18,7 @@ impl SendMessageOutput { match self { SendMessageOutput::Codewhisperer(output) => output.request_id(), SendMessageOutput::QDeveloper(output) => output.request_id(), + SendMessageOutput::Bedrock(output) => output.request_id(), SendMessageOutput::Mock(_) => None, } } @@ -29,6 +31,83 @@ impl SendMessageOutput { .await? .map(|s| s.into())), SendMessageOutput::QDeveloper(output) => Ok(output.send_message_response.recv().await?.map(|s| s.into())), + SendMessageOutput::Bedrock(output) => { + use aws_sdk_bedrockruntime::types::ConverseStreamOutput as BedrockStream; + use crate::api_client::error::{ConverseStreamError, ConverseStreamErrorKind, ConverseStreamSdkError}; + + let event = output.stream.recv().await + .map_err(|e| ApiClientError::ConverseStream( + ConverseStreamError::new( + ConverseStreamErrorKind::Unknown { + reason_code: e.to_string(), + }, + None::, + ) + ))?; + + match event { + Some(event) => match event { + BedrockStream::ContentBlockDelta(delta) => { + if let Some(delta_content) = delta.delta { + use aws_sdk_bedrockruntime::types::ContentBlockDelta; + match delta_content { + ContentBlockDelta::Text(text) => { + Ok(Some(ChatResponseStream::AssistantResponseEvent { + content: text, + })) + } + ContentBlockDelta::ToolUse(tool_use) => { + Ok(Some(ChatResponseStream::ToolUseEvent { + tool_use_id: delta.content_block_index.to_string(), + name: String::new(), + input: Some(tool_use.input), + stop: None, + })) + } + _ => Ok(Some(ChatResponseStream::Unknown)), + } + } else { + Ok(Some(ChatResponseStream::Unknown)) + } + } + BedrockStream::ContentBlockStart(start) => { + if let Some(start_content) = start.start { + use aws_sdk_bedrockruntime::types::ContentBlockStart; + match start_content { + ContentBlockStart::ToolUse(tool_use) => { + Ok(Some(ChatResponseStream::ToolUseEvent { + tool_use_id: tool_use.tool_use_id, + name: tool_use.name, + input: None, + stop: None, + })) + } + _ => Ok(Some(ChatResponseStream::Unknown)), + } + } else { + Ok(Some(ChatResponseStream::Unknown)) + } + } + BedrockStream::ContentBlockStop(_) => { + Ok(Some(ChatResponseStream::Unknown)) + } + BedrockStream::MessageStart(_) => { + Ok(Some(ChatResponseStream::Unknown)) + } + BedrockStream::MessageStop(_) => { + Ok(None) + } + BedrockStream::Metadata(metadata) => { + Ok(Some(ChatResponseStream::MessageMetadataEvent { + conversation_id: None, + utterance_id: metadata.usage.map(|u| format!("{:?}", u)), + })) + } + _ => Ok(Some(ChatResponseStream::Unknown)), + }, + None => Ok(None), + } + } SendMessageOutput::Mock(vec) => Ok(vec.pop()), } } @@ -39,6 +118,7 @@ impl RequestId for SendMessageOutput { match self { SendMessageOutput::Codewhisperer(output) => output.request_id(), SendMessageOutput::QDeveloper(output) => output.request_id(), + SendMessageOutput::Bedrock(output) => output.request_id(), SendMessageOutput::Mock(_) => Some(""), } } diff --git a/crates/chat-cli/src/cli/chat/cli/model.rs b/crates/chat-cli/src/cli/chat/cli/model.rs index 8ffcb8b70c..630e9f0713 100644 --- a/crates/chat-cli/src/cli/chat/cli/model.rs +++ b/crates/chat-cli/src/cli/chat/cli/model.rs @@ -171,6 +171,19 @@ pub async fn get_model_info(model_id: &str, os: &Os) -> Result Result<(Vec, ModelInfo), ChatError> { + // Check if Bedrock mode is enabled + let bedrock_enabled = os + .database + .settings + .get(crate::database::settings::Setting::BedrockEnabled) + .and_then(|v| v.as_bool()) + .unwrap_or(false); + + if bedrock_enabled { + // Get models from Bedrock + return get_bedrock_models(os).await; + } + let endpoint = Endpoint::configured_value(&os.database); let region = endpoint.region().as_ref(); @@ -204,6 +217,66 @@ fn default_context_window() -> usize { 200_000 } +async fn get_bedrock_models(os: &Os) -> Result<(Vec, ModelInfo), ChatError> { + use crate::api_client::bedrock::BedrockApiClient; + + let bedrock_client = BedrockApiClient::new(os.database.clone()) + .await + .map_err(|e| ChatError::Custom(format!("Failed to create Bedrock client: {}", e).into()))?; + + match bedrock_client.list_foundation_models().await { + Ok(model_ids) => { + let models: Vec = model_ids + .into_iter() + .map(|id| ModelInfo { + model_name: Some(id.clone()), + model_id: id, + description: None, + context_window_tokens: 200_000, + }) + .collect(); + + if models.is_empty() { + let fallback = get_bedrock_fallback_models(); + let default = fallback[0].clone(); + Ok((fallback, default)) + } else { + let default = models[0].clone(); + Ok((models, default)) + } + } + Err(e) => { + tracing::error!("Failed to fetch Bedrock models: {}, using fallback", e); + let fallback = get_bedrock_fallback_models(); + let default = fallback[0].clone(); + Ok((fallback, default)) + } + } +} + +fn get_bedrock_fallback_models() -> Vec { + vec![ + ModelInfo { + model_name: Some("Claude 3.5 Sonnet v2".to_string()), + model_id: "anthropic.claude-3-5-sonnet-20241022-v2:0".to_string(), + description: Some("Most intelligent model".to_string()), + context_window_tokens: 200_000, + }, + ModelInfo { + model_name: Some("Claude 3.5 Sonnet".to_string()), + model_id: "anthropic.claude-3-5-sonnet-20240620-v1:0".to_string(), + description: Some("Balance of intelligence and speed".to_string()), + context_window_tokens: 200_000, + }, + ModelInfo { + model_name: Some("Claude 3 Sonnet".to_string()), + model_id: "anthropic.claude-3-sonnet-20240229-v1:0".to_string(), + description: Some("Fast and efficient".to_string()), + context_window_tokens: 200_000, + }, + ] +} + fn get_fallback_models() -> Vec { vec![ ModelInfo { diff --git a/crates/chat-cli/src/cli/chat/mod.rs b/crates/chat-cli/src/cli/chat/mod.rs index 81043fdad3..c31873ee36 100644 --- a/crates/chat-cli/src/cli/chat/mod.rs +++ b/crates/chat-cli/src/cli/chat/mod.rs @@ -381,6 +381,20 @@ impl ChatArgs { let (models, default_model_opt) = get_available_models(os).await?; // Fallback logic: try user's saved default, then system default let fallback_model_id = || { + // Check if Bedrock mode is enabled + if os + .database + .settings + .get(Setting::BedrockEnabled) + .and_then(|v| v.as_bool()) + .unwrap_or(false) + { + // Use Bedrock model setting + if let Some(bedrock_model) = os.database.settings.get_string(Setting::BedrockModel) { + return Some(bedrock_model); + } + } + if let Some(saved) = os.database.settings.get_string(Setting::ChatDefaultModel) { find_model(&models, &saved) .map(|m| m.model_id.clone()) diff --git a/crates/chat-cli/src/cli/config.rs b/crates/chat-cli/src/cli/config.rs new file mode 100644 index 0000000000..26681d26a8 --- /dev/null +++ b/crates/chat-cli/src/cli/config.rs @@ -0,0 +1,265 @@ +use std::process::ExitCode; + +use anstream::println; +use clap::{Args, Subcommand, ValueEnum}; +use eyre::Result; +use serde_json::json; + +use crate::database::settings::Setting; +use crate::os::Os; + +#[derive(Clone, Debug, ValueEnum, PartialEq)] +pub enum BoolValue { + True, + False, +} + +impl BoolValue { + fn as_bool(&self) -> bool { + matches!(self, BoolValue::True) + } +} + +#[derive(Clone, Debug, Subcommand, PartialEq)] +pub enum ConfigSubcommand { + /// Enable or disable Bedrock backend mode + Bedrock { + /// Enable or disable Bedrock mode (true/false) + #[arg(value_enum)] + enable: BoolValue, + }, + /// Set AWS region for Bedrock + Region { + /// AWS region name (e.g., us-west-2) + region: String, + }, + /// Set context window size + ContextWindow { + /// Context window size (e.g., 8192) + size: u32, + }, + /// Set maximum output tokens + MaxTokens { + /// Maximum output tokens (e.g., 4096, max 200000) + tokens: u32, + }, + /// Select a Bedrock model interactively + BedrockModel, + /// Enable or disable extended thinking mode + Thinking { + /// Enable or disable thinking mode (true/false) + #[arg(value_enum)] + enabled: BoolValue, + }, + /// Set temperature for model responses + Temperature { + /// Temperature value between 0.0 and 1.0 + value: f64, + }, + /// Manage custom system prompts + #[command(subcommand)] + SystemPrompt(SystemPromptCommand), +} + +#[derive(Clone, Debug, Subcommand, PartialEq)] +pub enum SystemPromptCommand { + /// Add a new custom system prompt + Add { + /// Name of the system prompt + name: String, + /// The prompt text + prompt: String, + }, + /// Enable a system prompt + Enable { + /// Name of the system prompt to enable + name: String, + }, + /// Delete a system prompt + Delete { + /// Name of the system prompt to delete + name: String, + }, + /// List all system prompts + List, +} + +#[derive(Clone, Debug, Args, PartialEq)] +pub struct ConfigArgs { + #[command(subcommand)] + pub cmd: ConfigSubcommand, +} + +impl ConfigArgs { + pub async fn execute(&self, os: &mut Os) -> Result { + match &self.cmd { + ConfigSubcommand::Bedrock { enable } => { + let enabled = enable.as_bool(); + os.database.settings.set(Setting::BedrockEnabled, json!(enabled)).await?; + println!("Bedrock mode {}", if enabled { "enabled" } else { "disabled" }); + Ok(ExitCode::SUCCESS) + }, + ConfigSubcommand::Region { region } => { + os.database.settings.set(Setting::BedrockRegion, json!(region)).await?; + println!("Bedrock region set to: {}", region); + Ok(ExitCode::SUCCESS) + }, + ConfigSubcommand::ContextWindow { size } => { + os.database.settings.set(Setting::BedrockContextWindow, json!(size)).await?; + println!("Context window set to: {}", size); + Ok(ExitCode::SUCCESS) + }, + ConfigSubcommand::MaxTokens { tokens } => { + if *tokens > 200000 { + return Err(eyre::eyre!("Maximum output tokens cannot exceed 200000")); + } + os.database.settings.set(Setting::BedrockMaxTokens, json!(tokens)).await?; + println!("Maximum output tokens set to: {}", tokens); + Ok(ExitCode::SUCCESS) + }, + ConfigSubcommand::BedrockModel => { + use crate::api_client::bedrock::BedrockApiClient; + use dialoguer::Select; + + // Create Bedrock client to list models + let bedrock_client = BedrockApiClient::new(os.database.clone()) + .await + .map_err(|e| eyre::eyre!("Failed to create Bedrock client: {}", e))?; + + // Get available models + let models = bedrock_client.list_foundation_models() + .await + .map_err(|e| eyre::eyre!("Failed to list models: {}", e))?; + + if models.is_empty() { + return Err(eyre::eyre!("No models available")); + } + + // Get current model + let current_model = os.database.settings.get(Setting::BedrockModel) + .and_then(|v| v.as_str()) + .map(|s| s.to_string()); + + // Find default selection + let default_index = current_model + .as_ref() + .and_then(|current| models.iter().position(|m| m == current)) + .unwrap_or(0); + + // Show selection dialog + let selection = Select::with_theme(&crate::util::dialoguer_theme()) + .with_prompt("Select a Bedrock model") + .items(&models) + .default(default_index) + .interact_opt() + .map_err(|e| eyre::eyre!("Failed to select model: {}", e))?; + + if let Some(index) = selection { + let selected_model = &models[index]; + os.database.settings.set(Setting::BedrockModel, json!(selected_model)).await?; + println!("Bedrock model set to: {}", selected_model); + } + + Ok(ExitCode::SUCCESS) + }, + ConfigSubcommand::Thinking { enabled } => { + let enabled_bool = enabled.as_bool(); + os.database.settings.set(Setting::BedrockThinkingEnabled, json!(enabled_bool)).await?; + if enabled_bool { + println!("Thinking mode enabled (temperature will be set to 1.0)"); + } else { + println!("Thinking mode disabled"); + } + Ok(ExitCode::SUCCESS) + }, + ConfigSubcommand::Temperature { value } => { + if !(0.0..=1.0).contains(value) { + return Err(eyre::eyre!("Temperature must be between 0.0 and 1.0")); + } + + let thinking_enabled = os + .database + .settings + .get(Setting::BedrockThinkingEnabled) + .and_then(|v| v.as_bool()) + .unwrap_or(false); + + if thinking_enabled { + println!("Warning: Thinking mode is enabled. Temperature is automatically set to 1.0"); + return Ok(ExitCode::SUCCESS); + } + + os.database.settings.set(Setting::BedrockTemperature, json!(value)).await?; + println!("Temperature set to: {}", value); + Ok(ExitCode::SUCCESS) + }, + ConfigSubcommand::SystemPrompt(cmd) => { + match cmd { + SystemPromptCommand::Add { name, prompt } => { + let key = format!("bedrock.systemPrompt.{}", name); + os.database.settings.set_raw(&key, json!(prompt)).await?; + println!("System prompt '{}' added", name); + Ok(ExitCode::SUCCESS) + }, + SystemPromptCommand::Enable { name } => { + let key = format!("bedrock.systemPrompt.{}", name); + if os.database.settings.get_raw(&key).is_none() { + return Err(eyre::eyre!("System prompt '{}' not found", name)); + } + os.database.settings.set(Setting::BedrockSystemPromptActive, json!(name)).await?; + println!("System prompt '{}' enabled", name); + Ok(ExitCode::SUCCESS) + }, + SystemPromptCommand::Delete { name } => { + let key = format!("bedrock.systemPrompt.{}", name); + os.database.settings.remove_raw(&key).await?; + + let active = os.database.settings.get(Setting::BedrockSystemPromptActive) + .and_then(|v| v.as_str()) + .map(|s| s.to_string()); + + if active.as_deref() == Some(name.as_str()) { + os.database.settings.remove(Setting::BedrockSystemPromptActive).await?; + println!("System prompt '{}' deleted and deactivated", name); + } else { + println!("System prompt '{}' deleted", name); + } + Ok(ExitCode::SUCCESS) + }, + SystemPromptCommand::List => { + let all_settings = os.database.settings.map(); + let prompts: Vec<_> = all_settings + .iter() + .filter(|(k, _)| k.starts_with("bedrock.systemPrompt.") && *k != "bedrock.systemPrompt.active") + .collect(); + + if prompts.is_empty() { + println!("No custom system prompts configured"); + } else { + let active = os.database.settings.get(Setting::BedrockSystemPromptActive) + .and_then(|v| v.as_str()) + .map(|s| s.to_string()); + + println!("Custom system prompts:"); + for (key, value) in prompts { + let name = key.strip_prefix("bedrock.systemPrompt.").unwrap(); + let is_active = active.as_deref() == Some(name); + let marker = if is_active { " (active)" } else { "" }; + println!(" - {}{}", name, marker); + if let Some(text) = value.as_str() { + let preview = if text.len() > 60 { + format!("{}...", &text[..60]) + } else { + text.to_string() + }; + println!(" {}", preview); + } + } + } + Ok(ExitCode::SUCCESS) + }, + } + }, + } + } +} diff --git a/crates/chat-cli/src/cli/mod.rs b/crates/chat-cli/src/cli/mod.rs index 4fb89ef475..a696550b6b 100644 --- a/crates/chat-cli/src/cli/mod.rs +++ b/crates/chat-cli/src/cli/mod.rs @@ -5,6 +5,7 @@ use crate::util::env_var::{ }; mod agent; pub mod chat; +mod config; mod debug; mod diagnostics; pub mod experiment; @@ -48,6 +49,7 @@ use tracing::{ }; use crate::cli::chat::ChatArgs; +use crate::cli::config::ConfigArgs; use crate::cli::mcp::McpSubcommand; use crate::cli::user::{ LoginArgs, @@ -110,6 +112,8 @@ pub enum RootSubcommand { /// Customize appearance & behavior #[command(alias("setting"))] Settings(settings::SettingsArgs), + /// Configure Bedrock settings + Config(ConfigArgs), /// Run diagnostic tests #[command(alias("diagnostics"))] Diagnostic(diagnostics::DiagnosticArgs), @@ -141,8 +145,16 @@ impl RootSubcommand { } pub async fn execute(self, os: &mut Os) -> Result { - // Check for auth on subcommands that require it. - if self.requires_auth() && !crate::auth::is_logged_in(&mut os.database).await { + // Check if Bedrock mode is enabled + let bedrock_enabled = os + .database + .settings + .get(crate::database::settings::Setting::BedrockEnabled) + .and_then(|v| v.as_bool()) + .unwrap_or(false); + + // Check for auth on subcommands that require it (skip if Bedrock mode is enabled) + if !bedrock_enabled && self.requires_auth() && !crate::auth::is_logged_in(&mut os.database).await { bail!( "You are not logged in, please log in with {}", StyledText::command(&format!("{CLI_BINARY_NAME} login")) @@ -170,6 +182,7 @@ impl RootSubcommand { Self::Whoami(args) => args.execute(os).await, Self::Profile => user::profile(os).await, Self::Settings(settings_args) => settings_args.execute(os).await, + Self::Config(config_args) => config_args.execute(os).await, Self::Issue(args) => args.execute(os).await, Self::Version { changelog } => Cli::print_version(changelog), Self::Chat(args) => args.execute(os).await, @@ -194,6 +207,7 @@ impl Display for RootSubcommand { Self::Whoami(_) => "whoami", Self::Profile => "profile", Self::Settings(_) => "settings", + Self::Config(_) => "config", Self::Diagnostic(_) => "diagnostic", Self::Issue(_) => "issue", Self::Version { .. } => "version", diff --git a/crates/chat-cli/src/database/settings.rs b/crates/chat-cli/src/database/settings.rs index 02aec64a9a..45943072a0 100644 --- a/crates/chat-cli/src/database/settings.rs +++ b/crates/chat-cli/src/database/settings.rs @@ -91,6 +91,24 @@ pub enum Setting { EnabledDelegate, #[strum(message = "Specify UI variant to use (string)")] UiMode, + + // Bedrock-specific settings + #[strum(message = "Enable Bedrock backend mode (boolean)")] + BedrockEnabled, + #[strum(message = "AWS region for Bedrock API calls (string)")] + BedrockRegion, + #[strum(message = "Bedrock model ID to use (string)")] + BedrockModel, + #[strum(message = "Context window size for Bedrock (number)")] + BedrockContextWindow, + #[strum(message = "Maximum output tokens for Bedrock responses (number, max 200000)")] + BedrockMaxTokens, + #[strum(message = "Enable extended thinking mode for Bedrock (boolean)")] + BedrockThinkingEnabled, + #[strum(message = "Temperature setting for Bedrock (0.0-1.0)")] + BedrockTemperature, + #[strum(message = "Active custom system prompt name (string)")] + BedrockSystemPromptActive, } impl AsRef for Setting { @@ -133,6 +151,16 @@ impl AsRef for Setting { Self::EnabledContextUsageIndicator => "chat.enableContextUsageIndicator", Self::EnabledDelegate => "chat.enableDelegate", Self::UiMode => "chat.uiMode", + + // Bedrock settings + Self::BedrockEnabled => "bedrock.enabled", + Self::BedrockRegion => "bedrock.region", + Self::BedrockModel => "bedrock.model", + Self::BedrockContextWindow => "bedrock.contextWindow", + Self::BedrockMaxTokens => "bedrock.maxTokens", + Self::BedrockThinkingEnabled => "bedrock.thinkingEnabled", + Self::BedrockTemperature => "bedrock.temperature", + Self::BedrockSystemPromptActive => "bedrock.systemPrompt.active", } } } @@ -183,6 +211,18 @@ impl TryFrom<&str> for Setting { "chat.enableCheckpoint" => Ok(Self::EnabledCheckpoint), "chat.enableContextUsageIndicator" => Ok(Self::EnabledContextUsageIndicator), "chat.uiMode" => Ok(Self::UiMode), + "chat.enableDelegate" => Ok(Self::EnabledDelegate), + + // Bedrock settings + "bedrock.enabled" => Ok(Self::BedrockEnabled), + "bedrock.region" => Ok(Self::BedrockRegion), + "bedrock.model" => Ok(Self::BedrockModel), + "bedrock.contextWindow" => Ok(Self::BedrockContextWindow), + "bedrock.maxTokens" => Ok(Self::BedrockMaxTokens), + "bedrock.thinkingEnabled" => Ok(Self::BedrockThinkingEnabled), + "bedrock.temperature" => Ok(Self::BedrockTemperature), + "bedrock.systemPrompt.active" => Ok(Self::BedrockSystemPromptActive), + _ => Err(DatabaseError::InvalidSetting(value.to_string())), } } @@ -229,17 +269,32 @@ impl Settings { self.0.get(key.as_ref()) } + pub fn get_raw(&self, key: &str) -> Option<&Value> { + self.0.get(key) + } + pub async fn set(&mut self, key: Setting, value: impl Into) -> Result<(), DatabaseError> { self.0.insert(key.to_string(), value.into()); self.save_to_file().await } + pub async fn set_raw(&mut self, key: &str, value: impl Into) -> Result<(), DatabaseError> { + self.0.insert(key.to_string(), value.into()); + self.save_to_file().await + } + pub async fn remove(&mut self, key: Setting) -> Result, DatabaseError> { let key = self.0.remove(key.as_ref()); self.save_to_file().await?; Ok(key) } + pub async fn remove_raw(&mut self, key: &str) -> Result, DatabaseError> { + let key = self.0.remove(key); + self.save_to_file().await?; + Ok(key) + } + pub fn get_bool(&self, key: Setting) -> Option { self.get(key).and_then(|value| value.as_bool()) } diff --git a/rust-toolchain.toml b/rust-toolchain.toml index 70e8447f21..7181bf35dd 100644 --- a/rust-toolchain.toml +++ b/rust-toolchain.toml @@ -1,5 +1,5 @@ [toolchain] -channel = "1.87.0" +channel = "1.88.0" profile = "minimal" components = ["rustfmt", "clippy"] targets = [