diff --git a/README.md b/README.md index 01ad58e..69dcacb 100644 --- a/README.md +++ b/README.md @@ -50,6 +50,10 @@ For your applications running on Crusoe Managed Kubernetes cluster, you can coll ### Identity & Security +[Crusoe Bastion Host](./crusoe-bastion-host/) + +A production-ready, click-to-deploy bastion host solution for secure access to private infrastructure on Crusoe Cloud. This solution provides a hardened jump server with SSH key-based authentication, session logging, automatic security updates, fail2ban intrusion prevention, and comprehensive management tools. Includes an interactive deployment script for easy setup and supports high availability configurations for production environments. + [Crusoe to Splunk HEC Log Forwarder](./crusoe-splunk-hec/README.md) Crusoe Cloud provides a 90-day history of who did what in your cloud, when, where, and with what result - also called [Crusoe Audit Logs](https://docs.crusoecloud.com/identity-and-security/audit-logs/index.html). This solution provides a sample Python tool to fetch those logs and forward them to a Splunk HTTP Event Collector (HEC). \ No newline at end of file diff --git a/crusoe-bastion-host/.gitignore b/crusoe-bastion-host/.gitignore new file mode 100644 index 0000000..6bc11ae --- /dev/null +++ b/crusoe-bastion-host/.gitignore @@ -0,0 +1,44 @@ +# Terraform files +*.tfstate +*.tfstate.* +*.tfvars +!terraform.tfvars.example +.terraform/ +.terraform.lock.hcl +crash.log +override.tf +override.tf.json +*_override.tf +*_override.tf.json + +# SSH keys +*.pem +*.key +id_rsa* +id_ed25519* + +# Backup files +*.backup +*.bak +backup-*.tfstate +*-backup.tar.gz + +# Logs +*.log + +# OS files +.DS_Store +Thumbs.db +._* + +# Editor files +*.swp +*.swo +*~ +.vscode/ +.idea/ + +# Temporary files +tmp/ +temp/ +*.tmp diff --git a/crusoe-bastion-host/README.md b/crusoe-bastion-host/README.md new file mode 100644 index 0000000..1f89f81 --- /dev/null +++ b/crusoe-bastion-host/README.md @@ -0,0 +1,541 @@ +# Crusoe Bastion Host + +A production-ready, click-to-deploy bastion host solution for Crusoe Cloud. This solution provides a hardened jump server for secure access to private infrastructure with comprehensive security features and management tools. + +## Table of Contents + +- [Overview](#overview) +- [What is a Bastion Host?](#what-is-a-bastion-host) +- [Features](#features) +- [Prerequisites](#prerequisites) +- [Quick Start](#quick-start) +- [Manual Deployment](#manual-deployment) +- [Configuration](#configuration) +- [Usage](#usage) +- [Management](#management) +- [High Availability](#high-availability) +- [Security Considerations](#security-considerations) +- [Troubleshooting](#troubleshooting) +- [Advanced Configuration](#advanced-configuration) + +## Overview + +This solution deploys a hardened bastion host on Crusoe Cloud using Terraform. The bastion host serves as a secure gateway for SSH access to private instances, implementing security best practices and providing comprehensive audit logging. + +## What is a Bastion Host? + +A bastion host (also known as a jump server or jump box) is a special-purpose server designed to be the primary access point from an external network to resources within a private network. It acts as a secure gateway that: + +- **Provides a single point of entry** for administrative access +- **Reduces attack surface** by limiting direct access to private resources +- **Enables centralized access control** and monitoring +- **Facilitates audit logging** of all access attempts and sessions + +### Key Benefits + +- ✅ **Enhanced Security**: Hardened configuration with minimal attack surface +- ✅ **Access Control**: Centralized management of user access +- ✅ **Audit Trail**: Complete logging of all SSH sessions +- ✅ **Compliance**: Meets security and compliance requirements +- ✅ **Network Isolation**: Private instances don't need public IPs + +## Features + +### Security Features + +- 🔐 **SSH Key-Based Authentication Only** - No password authentication +- 🛡️ **Hardened SSH Configuration** - Modern ciphers and security settings +- 🚫 **Root Login Disabled** - Prevents direct root access +- 🔒 **Fail2ban Integration** - Automatic IP blocking after failed attempts +- 📝 **Session Logging** - Records all SSH sessions for audit purposes +- 🔄 **Automatic Security Updates** - Keeps system patched and secure +- 🔥 **UFW Firewall** - Restricts network access to essential services +- ⏱️ **Session Timeouts** - Automatic disconnection of idle sessions + +### Management Features + +- 👥 **User Management Scripts** - Easy addition/removal of users +- 📊 **Health Check Script** - Monitor bastion host status +- 📋 **Audit Log Viewer** - Review access logs and session recordings +- 🔧 **Infrastructure as Code** - Reproducible deployments with Terraform +- 🎯 **High Availability Option** - Deploy multiple bastions for redundancy + +### Deployment Features + +- 🚀 **Interactive Deployment Script** - Guided setup process +- ⚙️ **Customizable Configuration** - Flexible options for different use cases +- 📦 **One-Command Deployment** - Quick and easy setup +- 🏷️ **Tagging Support** - Organize resources with custom tags + +## Prerequisites + +Before deploying the bastion host, ensure you have: + +1. **Crusoe Cloud Account** with an active project +2. **Terraform** (>= 1.0) - [Install Terraform](https://www.terraform.io/downloads) +3. **Crusoe CLI** (optional but recommended) - [Install Crusoe CLI](https://docs.crusoecloud.com/quickstart/installing-the-cli/) +4. **SSH Key Pair** - For authentication to the bastion host +5. **jq** (optional) - For JSON parsing in scripts + +### Install Prerequisites + +```bash +# macOS +brew install terraform jq + +# Ubuntu/Debian +sudo apt-get install terraform jq + +# Verify installations +terraform version +jq --version +``` + +## Quick Start + +The fastest way to deploy a bastion host is using the interactive deployment script: + +```bash +# Clone the repository +git clone https://github.com/crusoecloud/solutions-library.git +cd solutions-library/crusoe-bastion-host + +# Make the deployment script executable +chmod +x deploy.sh + +# Run the interactive deployment +./deploy.sh +``` + +The script will guide you through: +1. ✓ Checking prerequisites +2. ✓ Collecting configuration (project ID, location, SSH key, etc.) +3. ✓ Configuring security features +4. ✓ Reviewing the configuration +5. ✓ Deploying with Terraform +6. ✓ Displaying connection instructions + +**That's it!** Your bastion host will be ready in a few minutes. + +## Manual Deployment + +If you prefer manual deployment or need more control: + +### 1. Configure Variables + +```bash +cd terraform +cp terraform.tfvars.example terraform.tfvars +``` + +Edit `terraform.tfvars` with your values: + +```hcl +project_id = "your-crusoe-project-id" +location = "us-east1-a" +ssh_public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC..." + +# Optional customizations +bastion_name = "bastion-host" +instance_type = "c1a.2x" +admin_username = "bastionadmin" + +allowed_ssh_cidrs = [ + "203.0.113.0/24", # Your office IP range +] + +enable_session_logging = true +auto_update_enabled = true +fail2ban_enabled = true +``` + +### 2. Initialize and Deploy + +```bash +# Initialize Terraform +terraform init + +# Review the deployment plan +terraform plan + +# Deploy the bastion host +terraform apply +``` + +### 3. Get Connection Information + +```bash +# View all outputs +terraform output + +# Get just the public IP +terraform output bastion_public_ips + +# Get SSH connection command +terraform output ssh_commands +``` + +## Configuration + +### Required Variables + +| Variable | Description | Example | +|----------|-------------|---------| +| `project_id` | Crusoe Cloud project ID | `"abc123..."` | +| `location` | Crusoe Cloud location | `"us-east1-a"` | +| `ssh_public_key` | SSH public key for access | `"ssh-rsa AAAA..."` | + +### Optional Variables + +| Variable | Default | Description | +|----------|---------|-------------| +| `bastion_name` | `"bastion-host"` | Name for the bastion instance | +| `instance_type` | `"c1a.2x"` | Instance type (2 vCPU, 4GB RAM) | +| `disk_size_gib` | `32` | Root disk size in GiB | +| `admin_username` | `"bastionadmin"` | Admin username | +| `allowed_ssh_cidrs` | `["0.0.0.0/0"]` | Allowed source IP ranges | +| `enable_session_logging` | `true` | Enable SSH session recording | +| `auto_update_enabled` | `true` | Enable automatic security updates | +| `fail2ban_enabled` | `true` | Enable fail2ban | +| `ssh_port` | `22` | SSH port (can be changed) | +| `session_timeout_seconds` | `900` | SSH session timeout (15 min) | +| `ha_enabled` | `false` | Enable high availability mode | +| `ha_count` | `2` | Number of bastions in HA mode | + +## Usage + +### Connecting to the Bastion Host + +```bash +# Direct SSH connection +ssh bastionadmin@ + +# Using SSH config (recommended) +# Add to ~/.ssh/config: +Host crusoe-bastion + HostName + User bastionadmin + IdentityFile ~/.ssh/your-private-key + ServerAliveInterval 60 + +# Then connect with: +ssh crusoe-bastion +``` + +### Accessing Private Instances via Bastion + +#### Method 1: SSH Jump Host (ProxyJump) + +```bash +# One-liner +ssh -J bastionadmin@ ubuntu@ + +# With SSH config +Host private-instance + HostName + User ubuntu + ProxyJump crusoe-bastion + IdentityFile ~/.ssh/your-private-key + +# Then connect with: +ssh private-instance +``` + +#### Method 2: SSH Tunneling + +```bash +# Create a tunnel +ssh -L 8080::80 bastionadmin@ + +# Access the private service +curl http://localhost:8080 +``` + +#### Method 3: SCP Through Bastion + +```bash +# Copy file to private instance +scp -o ProxyJump=bastionadmin@ file.txt ubuntu@:~/ + +# Copy file from private instance +scp -o ProxyJump=bastionadmin@ ubuntu@:~/file.txt . +``` + +## Management + +### Adding Users + +```bash +cd scripts + +# Add a new user with their SSH public key +./add-user.sh john ~/.ssh/john_id_rsa.pub + +# The script will: +# 1. Create the user account +# 2. Configure their SSH key +# 3. Update SSH configuration +# 4. Reload SSH service +``` + +### Removing Users + +```bash +cd scripts + +# Remove a user +./remove-user.sh john + +# This will: +# 1. Delete the user account +# 2. Remove their home directory +# 3. Update SSH configuration +``` + +### Viewing Audit Logs + +```bash +cd scripts + +# View recent audit logs +./audit-logs.sh + +# This displays: +# - SSH authentication logs +# - Active sessions +# - Recent login history +# - Session recordings +# - Fail2ban status +``` + +### Health Checks + +```bash +cd scripts + +# Run a comprehensive health check +./health-check.sh + +# Checks: +# - SSH connectivity +# - System resources (CPU, memory, disk) +# - Service status (SSH, UFW, fail2ban) +# - Active sessions +# - Security configuration +# - Available updates +# - Failed login attempts +``` + +### Testing Security Features + +The bastion host includes an automated test script to verify all security features are properly configured. This is useful after deployment or when troubleshooting. + +> **Note**: Wait 2-3 minutes after deployment before running tests to ensure all services (fail2ban, UFW, etc.) are fully initialized. + +```bash +cd scripts + +# Run all tests (remote connectivity + on-bastion security checks) +BASTION_IP= ./test-bastion.sh all + +# Run only remote tests (from your local machine) +BASTION_IP= ./test-bastion.sh remote + +# Run only bastion tests (must be run on the bastion itself) +./test-bastion.sh bastion +``` + +The test script validates: +- **SSH Connectivity** - Verifies the bastion is reachable +- **Root Login Disabled** - Confirms root SSH access is blocked +- **SSH Hardening** - Checks secure SSH configuration +- **Fail2ban** - Verifies intrusion prevention is active +- **UFW Firewall** - Confirms firewall rules are in place +- **Session Logging** - Validates session recording is configured +- **Admin User** - Checks bastionadmin account exists with proper permissions +- **Ubuntu User Disabled** - Confirms default user is locked (security hardening) + +## High Availability + +For production environments, deploy multiple bastion hosts for redundancy: + +```hcl +# In terraform.tfvars +ha_enabled = true +ha_count = 2 # Deploy 2 bastion hosts +``` + +This creates multiple bastion hosts that can be used with: + +- **DNS Round-Robin**: Point a DNS record to all bastion IPs +- **Load Balancer**: Use a TCP load balancer (Layer 4) +- **Client-Side Failover**: Configure multiple ProxyJump hosts in SSH config + +### SSH Config for HA + +``` +# ~/.ssh/config +Host crusoe-bastion-1 + HostName + User bastionadmin + +Host crusoe-bastion-2 + HostName + User bastionadmin + +Host private-instance + HostName + User ubuntu + ProxyJump crusoe-bastion-1,crusoe-bastion-2 +``` + +## Security Considerations + +### Best Practices + +1. **Restrict Source IPs**: Limit `allowed_ssh_cidrs` to known IP ranges +2. **Regular Key Rotation**: Rotate SSH keys periodically +3. **Monitor Logs**: Review audit logs regularly +4. **Keep Updated**: Ensure automatic updates are enabled +5. **Minimal Privileges**: Grant users only necessary access +6. **Session Recording**: Keep session logs for compliance +7. **Network Segmentation**: Use private subnets for internal resources + +### Security Checklist + +- ✅ SSH key-based authentication only +- ✅ Root login disabled +- ✅ Password authentication disabled +- ✅ Fail2ban enabled and configured +- ✅ UFW firewall active +- ✅ Automatic security updates enabled +- ✅ Session logging enabled +- ✅ SSH hardening configuration applied +- ✅ Source IP restrictions configured +- ✅ Session timeouts configured + +For detailed security information, see [SECURITY.md](./SECURITY.md). + +## Troubleshooting + +### Cannot Connect to Bastion + +**Problem**: SSH connection times out or is refused + +**Solutions**: +1. Verify the bastion is running: `crusoe compute vms list` +2. Check firewall rules allow your IP: Review `allowed_ssh_cidrs` +3. Verify SSH key is correct: `ssh-add -l` +4. Check bastion logs: `./scripts/audit-logs.sh ` + +### User Cannot Login + +**Problem**: User gets "Permission denied" error + +**Solutions**: +1. Verify user was added: `./scripts/audit-logs.sh ` +2. Check SSH key format: Must be valid public key +3. Verify user in AllowUsers: SSH config must include username +4. Check fail2ban: User's IP may be banned + +### Session Disconnects + +**Problem**: SSH session disconnects after idle time + +**Solutions**: +1. This is expected behavior (security feature) +2. Adjust `session_timeout_seconds` if needed +3. Use `ServerAliveInterval` in SSH config: + ``` + Host crusoe-bastion + ServerAliveInterval 60 + ServerAliveCountMax 3 + ``` + +### Cannot Access Private Instance + +**Problem**: Cannot SSH to private instance through bastion + +**Solutions**: +1. Verify bastion can reach private instance: + ```bash + ssh bastionadmin@ + ping + ``` +2. Check private instance firewall allows bastion's private IP +3. Verify SSH key is available on bastion or use agent forwarding: + ```bash + ssh -A bastionadmin@ + ``` + +### Terraform Errors + +**Problem**: Terraform apply fails + +**Solutions**: +1. Verify Crusoe credentials: `crusoe config list` +2. Check project ID is correct +3. Verify location is valid: `crusoe locations list` +4. Review Terraform logs: `TF_LOG=DEBUG terraform apply` + +## Advanced Configuration + +### Custom SSH Port + +```hcl +# terraform.tfvars +ssh_port = 2222 # Use non-standard port +``` + +Update firewall rules accordingly. + +### Custom Hardening Script + +Modify `terraform/user-data.sh` to add custom security configurations. + +### Integration with Monitoring + +Add monitoring agents in the user-data script: + +```bash +# Install monitoring agent +apt-get install -y datadog-agent +# Configure agent... +``` + +### Backup Configuration + +```bash +# Backup Terraform state +terraform state pull > backup-$(date +%Y%m%d).tfstate + +# Backup SSH keys and configs +tar -czf bastion-backup-$(date +%Y%m%d).tar.gz \ + terraform/terraform.tfvars \ + ~/.ssh/config +``` + +## Cleanup + +To remove the bastion host and all resources: + +```bash +cd terraform +terraform destroy +``` + +**Warning**: This will permanently delete the bastion host and all associated resources. + +## Support + +- **Documentation**: [Crusoe Cloud Docs](https://docs.crusoecloud.com/) +- **Issues**: [GitHub Issues](https://github.com/crusoecloud/solutions-library/issues) +- **Security**: See [SECURITY.md](./SECURITY.md) for security policies + +## License + +This solution is provided as-is for use with Crusoe Cloud. + +## Contributing + +Contributions are welcome! Please submit pull requests or issues to the [solutions-library repository](https://github.com/crusoecloud/solutions-library). diff --git a/crusoe-bastion-host/SECURITY.md b/crusoe-bastion-host/SECURITY.md new file mode 100644 index 0000000..32c5fba --- /dev/null +++ b/crusoe-bastion-host/SECURITY.md @@ -0,0 +1,649 @@ +# Security Best Practices for Bastion Hosts + +This document outlines security best practices, hardening guidelines, and compliance considerations for the Crusoe Bastion Host solution. + +## Table of Contents + +- [Security Architecture](#security-architecture) +- [Hardening Checklist](#hardening-checklist) +- [Access Control](#access-control) +- [Network Security](#network-security) +- [Audit and Compliance](#audit-and-compliance) +- [Incident Response](#incident-response) +- [Security Maintenance](#security-maintenance) +- [Compliance Frameworks](#compliance-frameworks) + +## Security Architecture + +### Defense in Depth + +The bastion host implements multiple layers of security: + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Layer 1: Network Security │ +│ - Firewall rules (UFW) │ +│ - Source IP restrictions │ +│ - Fail2ban intrusion prevention │ +└─────────────────────────────────────────────────────────────┘ + ↓ +┌─────────────────────────────────────────────────────────────┐ +│ Layer 2: Authentication │ +│ - SSH key-based authentication only │ +│ - No password authentication │ +│ - Root login disabled │ +└─────────────────────────────────────────────────────────────┘ + ↓ +┌─────────────────────────────────────────────────────────────┐ +│ Layer 3: Authorization │ +│ - User-based access control │ +│ - AllowUsers SSH restriction │ +│ - Sudo privileges management │ +└─────────────────────────────────────────────────────────────┘ + ↓ +┌─────────────────────────────────────────────────────────────┐ +│ Layer 4: Monitoring & Auditing │ +│ - Session logging and recording │ +│ - Authentication logs │ +│ - Failed login attempt tracking │ +└─────────────────────────────────────────────────────────────┘ +``` + +### Security Principles + +1. **Least Privilege**: Users have minimal necessary permissions +2. **Zero Trust**: Verify every access attempt +3. **Defense in Depth**: Multiple security layers +4. **Audit Everything**: Comprehensive logging +5. **Fail Secure**: Default deny policies + +## Hardening Checklist + +### SSH Hardening + +- ✅ **Protocol 2 Only**: Modern SSH protocol +- ✅ **Key-Based Authentication**: No passwords allowed +- ✅ **Root Login Disabled**: `PermitRootLogin no` +- ✅ **Empty Passwords Disabled**: `PermitEmptyPasswords no` +- ✅ **X11 Forwarding Disabled**: Reduces attack surface +- ✅ **Max Auth Tries Limited**: `MaxAuthTries 3` +- ✅ **Strong Ciphers**: Modern encryption algorithms +- ✅ **Session Timeouts**: Automatic disconnection +- ✅ **Login Banner**: Legal warning message + +### SSH Configuration Details + +``` +# /etc/ssh/sshd_config.d/99-bastion-hardening.conf + +# Authentication +PermitRootLogin no +PubkeyAuthentication yes +PasswordAuthentication no +ChallengeResponseAuthentication no + +# Security +PermitEmptyPasswords no +X11Forwarding no +MaxAuthTries 3 +MaxSessions 10 + +# Timeouts +ClientAliveInterval 300 +ClientAliveCountMax 2 + +# Logging +LogLevel VERBOSE +SyslogFacility AUTH + +# Cryptography +Protocol 2 +Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com +MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com +KexAlgorithms curve25519-sha256,diffie-hellman-group-exchange-sha256 +``` + +### System Hardening + +- ✅ **Automatic Security Updates**: Unattended-upgrades configured +- ✅ **Minimal Package Installation**: Only essential packages +- ✅ **Kernel Hardening**: Sysctl security parameters +- ✅ **Firewall Enabled**: UFW with default deny +- ✅ **Fail2ban Active**: Automatic IP blocking +- ✅ **Audit Logging**: Session recording enabled + +### Kernel Security Parameters + +```bash +# /etc/sysctl.conf additions + +# Disable IP forwarding (bastion is not a router) +net.ipv4.ip_forward = 0 + +# Disable source routing +net.ipv4.conf.all.accept_source_route = 0 +net.ipv4.conf.default.accept_source_route = 0 + +# Disable ICMP redirects +net.ipv4.conf.all.accept_redirects = 0 +net.ipv4.conf.default.accept_redirects = 0 +net.ipv4.conf.all.secure_redirects = 0 +net.ipv4.conf.default.secure_redirects = 0 +net.ipv4.conf.all.send_redirects = 0 +net.ipv4.conf.default.send_redirects = 0 + +# Enable reverse path filtering +net.ipv4.conf.all.rp_filter = 1 +net.ipv4.conf.default.rp_filter = 1 + +# Ignore ICMP broadcasts +net.ipv4.icmp_echo_ignore_broadcasts = 1 + +# Ignore bogus ICMP errors +net.ipv4.icmp_ignore_bogus_error_responses = 1 + +# Enable TCP SYN cookies +net.ipv4.tcp_syncookies = 1 +``` + +## Access Control + +### User Management Best Practices + +#### 1. SSH Key Management + +**Generate Strong Keys**: +```bash +# Recommended: Ed25519 (modern, secure, fast) +ssh-keygen -t ed25519 -C "user@example.com" + +# Alternative: RSA 4096-bit +ssh-keygen -t rsa -b 4096 -C "user@example.com" +``` + +**Key Rotation Policy**: +- Rotate SSH keys every 90-180 days +- Immediately revoke keys for departed users +- Use different keys for different environments +- Never share private keys + +**Key Storage**: +- Store private keys securely (encrypted disk, password manager) +- Use SSH agent for key management +- Consider hardware security keys (YubiKey, etc.) + +#### 2. User Lifecycle + +**Adding Users**: +```bash +# Use the provided script +./scripts/add-user.sh username ~/.ssh/username_key.pub + +# Verify user was added +./scripts/audit-logs.sh +``` + +**Removing Users**: +```bash +# Remove immediately when access is no longer needed +./scripts/remove-user.sh username + +# Verify removal +ssh username@ # Should fail +``` + +**Regular Access Reviews**: +- Review user list monthly +- Remove inactive users +- Verify users still require access +- Document access justifications + +#### 3. Privilege Management + +**Sudo Access**: +```bash +# Grant sudo only when necessary +echo "username ALL=(ALL) NOPASSWD:/specific/command" > /etc/sudoers.d/username + +# Avoid blanket sudo access +# Review sudo logs regularly +``` + +**Principle of Least Privilege**: +- Users should only access what they need +- Time-bound access for temporary needs +- Separate accounts for different roles + +### Multi-Factor Authentication (MFA) + +For enhanced security, consider implementing MFA: + +**Google Authenticator**: +```bash +# Install on bastion +apt-get install libpam-google-authenticator + +# Configure PAM +# Edit /etc/pam.d/sshd +auth required pam_google_authenticator.so + +# Update SSH config +# /etc/ssh/sshd_config +ChallengeResponseAuthentication yes +AuthenticationMethods publickey,keyboard-interactive +``` + +**Hardware Keys (YubiKey)**: +```bash +# Install required packages +apt-get install libpam-u2f + +# Configure for SSH +# Users enroll their hardware keys +pamu2fcfg > ~/.config/Yubico/u2f_keys +``` + +## Network Security + +### Firewall Configuration + +**UFW Rules**: +```bash +# Default policies +ufw default deny incoming +ufw default allow outgoing + +# Allow SSH from specific IPs only +ufw allow from 203.0.113.0/24 to any port 22 + +# Enable firewall +ufw enable + +# Check status +ufw status verbose +``` + +**Advanced Firewall Rules**: +```bash +# Rate limiting SSH connections +ufw limit 22/tcp + +# Allow from specific IP only +ufw allow from 203.0.113.10 to any port 22 + +# Deny from known bad actors +ufw deny from 198.51.100.0/24 +``` + +### Fail2ban Configuration + +**SSH Jail Settings**: +```ini +# /etc/fail2ban/jail.local + +[sshd] +enabled = true +port = 22 +filter = sshd +logpath = /var/log/auth.log +maxretry = 3 +bantime = 7200 # 2 hours +findtime = 600 # 10 minutes +action = iptables[name=SSH, port=22, protocol=tcp] +``` + +**Monitor Fail2ban**: +```bash +# Check status +fail2ban-client status sshd + +# View banned IPs +fail2ban-client get sshd banned + +# Unban an IP +fail2ban-client set sshd unbanip +``` + +### Network Segmentation + +**Best Practices**: +1. **Public Subnet**: Bastion host only +2. **Private Subnets**: Internal resources +3. **Security Groups**: Restrict traffic between subnets +4. **No Direct Internet**: Private instances have no public IPs + +**Firewall Rules for Private Instances**: +```bash +# On private instances, allow SSH only from bastion +ufw allow from to any port 22 +ufw default deny incoming +``` + +## Audit and Compliance + +### Session Logging + +**What is Logged**: +- SSH connection attempts (successful and failed) +- User authentication events +- Commands executed (with session recording) +- Session start/end times +- Source IP addresses + +**Log Locations**: +``` +/var/log/auth.log # Authentication logs +/var/log/bastion-sessions/ # Session recordings +/var/log/syslog # System logs +``` + +**Viewing Logs**: +```bash +# Recent SSH attempts +grep "sshd" /var/log/auth.log | tail -50 + +# Failed login attempts +grep "Failed password" /var/log/auth.log + +# Successful logins +grep "Accepted publickey" /var/log/auth.log + +# Active sessions +who +last -n 20 +``` + +### Session Recording + +Session recordings capture all terminal activity: + +```bash +# List session recordings +ls -lh /var/log/bastion-sessions/ + +# View a session recording +cat /var/log/bastion-sessions/20260121-143000-user-12345.log + +# Download session recording +scp bastion:/var/log/bastion-sessions/session.log . +``` + +### Log Retention + +**Recommended Retention Periods**: +- **Authentication logs**: 90 days minimum +- **Session recordings**: 1 year for compliance +- **System logs**: 30 days + +**Implement Log Rotation**: +```bash +# /etc/logrotate.d/bastion-sessions +/var/log/bastion-sessions/*.log { + daily + rotate 365 + compress + delaycompress + missingok + notifempty +} +``` + +### Centralized Logging + +Forward logs to a centralized system: + +**Rsyslog Configuration**: +```bash +# /etc/rsyslog.d/50-bastion.conf +*.* @@log-server.example.com:514 +``` + +**Integration Options**: +- Splunk (see [crusoe-splunk-hec](../crusoe-splunk-hec/)) +- ELK Stack (Elasticsearch, Logstash, Kibana) +- Cloud logging (GCP Cloud Logging, AWS CloudWatch) +- Syslog servers + +## Incident Response + +### Detecting Security Incidents + +**Warning Signs**: +- Multiple failed login attempts +- Logins from unexpected locations +- Unusual command execution patterns +- Unexpected system changes +- High resource usage +- New user accounts + +**Monitoring Commands**: +```bash +# Check for suspicious activity +./scripts/audit-logs.sh + +# Monitor in real-time +ssh bastion 'sudo tail -f /var/log/auth.log' + +# Check fail2ban bans +ssh bastion 'sudo fail2ban-client status sshd' +``` + +### Incident Response Procedures + +**1. Detection**: +```bash +# Run health check +./scripts/health-check.sh + +# Review recent logs +./scripts/audit-logs.sh +``` + +**2. Containment**: +```bash +# Block suspicious IP immediately +ssh bastion 'sudo ufw deny from ' + +# Disable compromised user +ssh bastion 'sudo usermod -L username' + +# Kill active sessions +ssh bastion 'sudo pkill -u username' +``` + +**3. Investigation**: +```bash +# Collect evidence +ssh bastion 'sudo tar -czf /tmp/evidence.tar.gz /var/log/auth.log /var/log/bastion-sessions/' +scp bastion:/tmp/evidence.tar.gz ./evidence-$(date +%Y%m%d).tar.gz + +# Review session recordings +# Analyze authentication logs +# Check for unauthorized changes +``` + +**4. Recovery**: +```bash +# Remove compromised user +./scripts/remove-user.sh compromised-user + +# Rotate SSH keys +# Update firewall rules +# Apply security patches + +# Verify system integrity +./scripts/health-check.sh +``` + +**5. Post-Incident**: +- Document the incident +- Update security procedures +- Implement additional controls +- Conduct lessons learned review + +### Emergency Procedures + +**Lockdown Mode**: +```bash +# Block all SSH access except from specific IP +ssh bastion 'sudo ufw default deny incoming' +ssh bastion 'sudo ufw allow from to any port 22' +ssh bastion 'sudo ufw reload' +``` + +**Emergency User Removal**: +```bash +# Immediately disable user +ssh bastion 'sudo usermod -L username && sudo pkill -u username' +``` + +## Security Maintenance + +### Regular Tasks + +**Daily**: +- Monitor authentication logs +- Check fail2ban status +- Review active sessions + +**Weekly**: +- Review user access list +- Check for available updates +- Verify backup integrity + +**Monthly**: +- Conduct access reviews +- Review and rotate logs +- Test incident response procedures +- Update documentation + +**Quarterly**: +- Rotate SSH keys +- Security audit +- Penetration testing +- Review and update security policies + +### Update Management + +**Automatic Updates**: +```bash +# Verify unattended-upgrades is active +systemctl status unattended-upgrades + +# Check update configuration +cat /etc/apt/apt.conf.d/50unattended-upgrades +``` + +**Manual Updates**: +```bash +# Check for updates +ssh bastion 'sudo apt update && apt list --upgradable' + +# Apply security updates +ssh bastion 'sudo apt upgrade -y' + +# Reboot if kernel updated +ssh bastion 'sudo reboot' +``` + +### Security Scanning + +**Vulnerability Scanning**: +```bash +# Install and run lynis +ssh bastion 'sudo apt install lynis' +ssh bastion 'sudo lynis audit system' + +# Check for outdated packages +ssh bastion 'sudo apt list --upgradable' +``` + +**Configuration Auditing**: +```bash +# SSH configuration test +ssh bastion 'sudo sshd -t' + +# Firewall status +ssh bastion 'sudo ufw status verbose' + +# Check for weak permissions +ssh bastion 'sudo find /home -type f -perm -002' +``` + +## Compliance Frameworks + +### SOC 2 Compliance + +**Requirements Met**: +- ✅ Access controls (CC6.1) +- ✅ Logical and physical access restrictions (CC6.2) +- ✅ Audit logging (CC7.2) +- ✅ Security monitoring (CC7.3) + +**Evidence Collection**: +- User access logs +- Session recordings +- Change management records +- Security configuration documentation + +### PCI DSS Compliance + +**Requirements Met**: +- ✅ Requirement 2: Secure configurations +- ✅ Requirement 8: User authentication +- ✅ Requirement 10: Audit trails +- ✅ Requirement 11: Security testing + +**Specific Controls**: +- Strong cryptography (Req 4) +- Multi-factor authentication (Req 8.3) +- Log retention (Req 10.7) + +### HIPAA Compliance + +**Technical Safeguards**: +- ✅ Access control (§164.312(a)(1)) +- ✅ Audit controls (§164.312(b)) +- ✅ Integrity controls (§164.312(c)(1)) +- ✅ Transmission security (§164.312(e)(1)) + +**Documentation**: +- Access authorization records +- Audit log reviews +- Security incident procedures +- Risk assessments + +### ISO 27001 Compliance + +**Controls Implemented**: +- A.9.2: User access management +- A.9.4: System and application access control +- A.12.4: Logging and monitoring +- A.18.1: Compliance with legal requirements + +## Security Resources + +### Tools + +- **SSH Audit**: https://github.com/jtesta/ssh-audit +- **Lynis**: https://cisofy.com/lynis/ +- **OSSEC**: https://www.ossec.net/ +- **Wazuh**: https://wazuh.com/ + +### References + +- **CIS Benchmarks**: https://www.cisecurity.org/cis-benchmarks/ +- **NIST Guidelines**: https://csrc.nist.gov/publications +- **OWASP**: https://owasp.org/ +- **SSH Hardening**: https://www.ssh.com/academy/ssh/security + +### Security Contacts + +For security issues or questions: +- Review the [README.md](./README.md) +- Check [Crusoe Cloud Documentation](https://docs.crusoecloud.com/) +- Report security vulnerabilities responsibly + +--- + +**Remember**: Security is an ongoing process, not a one-time setup. Regular monitoring, updates, and reviews are essential for maintaining a secure bastion host. diff --git a/crusoe-bastion-host/deploy.sh b/crusoe-bastion-host/deploy.sh new file mode 100755 index 0000000..05b8740 --- /dev/null +++ b/crusoe-bastion-host/deploy.sh @@ -0,0 +1,535 @@ +#!/bin/bash +set -e + +# Interactive Bastion Host Deployment Script +# This script guides users through deploying a bastion host on Crusoe Cloud + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +TERRAFORM_DIR="$SCRIPT_DIR/terraform" +TFVARS_FILE="$TERRAFORM_DIR/terraform.tfvars" + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +NC='\033[0m' # No Color + +# Print functions +print_header() { + echo "" + echo -e "${CYAN}╔════════════════════════════════════════════════════════════════════════════╗${NC}" + echo -e "${CYAN}║${NC} ${BLUE}CRUSOE BASTION HOST DEPLOYMENT${NC} ${CYAN}║${NC}" + echo -e "${CYAN}╚════════════════════════════════════════════════════════════════════════════╝${NC}" + echo "" +} + +print_section() { + echo "" + echo -e "${CYAN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" + echo -e "${BLUE}$1${NC}" + echo -e "${CYAN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +print_success() { + echo -e "${GREEN}✓${NC} $1" +} + +print_error() { + echo -e "${RED}✗${NC} $1" +} + +print_warning() { + echo -e "${YELLOW}⚠${NC} $1" +} + +print_info() { + echo -e "${CYAN}ℹ${NC} $1" +} + +# Check prerequisites +check_prerequisites() { + print_section "Checking Prerequisites" + + local all_good=true + + # Check Terraform + if command -v terraform &> /dev/null; then + TERRAFORM_VERSION=$(terraform version -json | grep -o '"terraform_version":"[^"]*' | cut -d'"' -f4) + print_success "Terraform installed (version $TERRAFORM_VERSION)" + else + print_error "Terraform is not installed" + print_info "Install from: https://www.terraform.io/downloads" + all_good=false + fi + + # Check Crusoe CLI + if command -v crusoe &> /dev/null; then + print_success "Crusoe CLI installed" + else + print_warning "Crusoe CLI not found (optional but recommended)" + print_info "Install from: https://docs.crusoecloud.com/quickstart/installing-the-cli/" + fi + + # Check jq + if command -v jq &> /dev/null; then + print_success "jq installed" + else + print_warning "jq not found (optional, used for JSON parsing)" + fi + + # Check SSH + if command -v ssh &> /dev/null; then + print_success "SSH client available" + else + print_error "SSH client not found" + all_good=false + fi + + if [ "$all_good" = false ]; then + echo "" + print_error "Please install missing prerequisites before continuing" + exit 1 + fi + + echo "" +} + +# Prompt for input with validation +prompt_input() { + local prompt="$1" + local var_name="$2" + local default="$3" + local validation="$4" + + while true; do + if [ -n "$default" ]; then + printf "\033[0;36m%s\033[0m [%s]: " "$prompt" "$default" + read input + input="${input:-$default}" + else + printf "\033[0;36m%s\033[0m: " "$prompt" + read input + fi + + # Validate input if validation function provided + if [ -n "$validation" ]; then + if $validation "$input"; then + eval "$var_name='$input'" + break + fi + else + if [ -n "$input" ]; then + eval "$var_name='$input'" + break + else + print_error "This field is required" + fi + fi + done +} + +# Validation functions +validate_project_id() { + if [[ "$1" =~ ^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$ ]]; then + return 0 + else + print_error "Invalid project ID format (expected UUID format)" + return 1 + fi +} + +validate_location() { + if [ -n "$1" ]; then + return 0 + else + print_error "Location is required" + return 1 + fi +} + +validate_ssh_key() { + if [ -f "$1" ]; then + if [[ $(cat "$1") =~ ^(ssh-rsa|ssh-ed25519|ecdsa-sha2-nistp) ]]; then + return 0 + else + print_error "File does not contain a valid SSH public key" + return 1 + fi + else + print_error "SSH key file not found: $1" + return 1 + fi +} + +validate_cidr() { + if [[ "$1" =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}/[0-9]{1,2}$ ]]; then + return 0 + elif [ "$1" = "done" ]; then + return 0 + else + print_error "Invalid CIDR format (e.g., 203.0.113.0/24)" + return 1 + fi +} + +# Fetch available VPC networks +fetch_vpc_networks() { + print_info "Fetching available VPC networks..." + + if ! command -v crusoe &> /dev/null; then + print_warning "Crusoe CLI not installed. Using default VPC network name." + VPC_NETWORK="default" + return + fi + + # Fetch VPC networks using Crusoe CLI + local vpc_json + vpc_json=$(crusoe networking vpc-networks list --project-id "$PROJECT_ID" --json 2>/dev/null) + + if [ $? -ne 0 ] || [ -z "$vpc_json" ]; then + print_warning "Could not fetch VPC networks. Using default." + VPC_NETWORK="default" + return + fi + + # Parse VPC names using jq if available + if command -v jq &> /dev/null; then + local vpc_names + vpc_names=$(echo "$vpc_json" | jq -r '.[].name' 2>/dev/null) + + if [ -z "$vpc_names" ]; then + print_warning "No VPC networks found. Using default." + VPC_NETWORK="default" + return + fi + + # Display available VPCs + echo "" + print_info "Available VPC Networks:" + local idx=1 + local vpc_array=() + while IFS= read -r vpc; do + echo " $idx) $vpc" + vpc_array+=("$vpc") + ((idx++)) + done <<< "$vpc_names" + + echo "" + printf "\033[0;36mSelect VPC network\033[0m [1]: " + read vpc_choice + vpc_choice="${vpc_choice:-1}" + + # Validate selection + if [[ "$vpc_choice" =~ ^[0-9]+$ ]] && [ "$vpc_choice" -ge 1 ] && [ "$vpc_choice" -le "${#vpc_array[@]}" ]; then + VPC_NETWORK="${vpc_array[$((vpc_choice-1))]}" + print_success "Selected VPC: $VPC_NETWORK" + else + print_warning "Invalid selection. Using first VPC: ${vpc_array[0]}" + VPC_NETWORK="${vpc_array[0]}" + fi + else + print_warning "jq not installed. Using default VPC network name." + VPC_NETWORK="default" + fi +} + +# Collect configuration +collect_configuration() { + print_section "Configuration" + + echo "Let's configure your bastion host deployment." + echo "" + + # Project ID - try to auto-detect from CLI + print_info "Detecting your Crusoe project..." + local detected_project="" + local detected_project_id="" + local detected_project_name="" + + if command -v crusoe &> /dev/null; then + # Get the default project from CLI config (could be name or ID) + detected_project=$(crusoe config get default_project 2>/dev/null || echo "") + + if [ -n "$detected_project" ]; then + # Check if it's already a UUID + if [[ "$detected_project" =~ ^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$ ]]; then + detected_project_id="$detected_project" + else + # It's a project name, try to get the ID + detected_project_name="$detected_project" + detected_project_id=$(crusoe projects list --json 2>/dev/null | jq -r --arg name "$detected_project" '.[] | select(.name == $name) | .id' 2>/dev/null || echo "") + fi + fi + fi + + if [ -n "$detected_project_id" ] && [[ "$detected_project_id" =~ ^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$ ]]; then + if [ -n "$detected_project_name" ]; then + print_success "Detected project: $detected_project_name ($detected_project_id)" + else + print_success "Detected project: $detected_project_id" + fi + echo "" + echo " 1) Use detected project" + echo " 2) Enter a different project ID" + echo "" + printf "\033[0;36mSelect option\033[0m [1]: " + read project_choice + project_choice="${project_choice:-1}" + + if [ "$project_choice" = "1" ]; then + PROJECT_ID="$detected_project_id" + else + prompt_input "Project ID" PROJECT_ID "" validate_project_id + fi + else + print_warning "Could not auto-detect project ID" + prompt_input "Project ID" PROJECT_ID "" validate_project_id + fi + + # Fetch and select VPC network + fetch_vpc_networks + + # Location + print_info "Available locations: us-east1-a, us-central1-a, etc." + prompt_input "Location" LOCATION "us-east1-a" validate_location + + # Bastion name + prompt_input "Bastion host name" BASTION_NAME "bastion-host" + + # Instance type + print_info "Recommended: c1a.2x (2 vCPU, 4GB RAM) for bastion hosts" + prompt_input "Instance type" INSTANCE_TYPE "c1a.2x" + + # SSH key + echo "" + print_info "Path to your SSH public key file" + print_info "Common locations: ~/.ssh/id_rsa.pub, ~/.ssh/id_ed25519.pub" + + # Try to find SSH keys + if [ -f ~/.ssh/id_ed25519.pub ]; then + DEFAULT_KEY=~/.ssh/id_ed25519.pub + elif [ -f ~/.ssh/id_rsa.pub ]; then + DEFAULT_KEY=~/.ssh/id_rsa.pub + else + DEFAULT_KEY="" + fi + + prompt_input "SSH public key path" SSH_KEY_PATH "$DEFAULT_KEY" validate_ssh_key + SSH_PUBLIC_KEY=$(cat "$SSH_KEY_PATH") + + # Allowed SSH CIDRs + echo "" + print_info "Restrict SSH access to specific IP ranges (CIDR notation)" + print_warning "Using 0.0.0.0/0 allows access from anywhere (not recommended for production)" + + read -p $'\033[0;36mRestrict SSH access to specific IPs?\033[0m (y/n) [n]: ' RESTRICT_SSH + RESTRICT_SSH="${RESTRICT_SSH:-n}" + + if [[ "$RESTRICT_SSH" =~ ^[Yy] ]]; then + ALLOWED_CIDRS="" + while true; do + prompt_input "Enter CIDR (or 'done' to finish)" CIDR "" validate_cidr + if [ "$CIDR" = "done" ]; then + break + fi + if [ -z "$ALLOWED_CIDRS" ]; then + ALLOWED_CIDRS="\"$CIDR\"" + else + ALLOWED_CIDRS="$ALLOWED_CIDRS, \"$CIDR\"" + fi + print_success "Added: $CIDR" + done + else + ALLOWED_CIDRS="\"0.0.0.0/0\"" + fi + + # High Availability + echo "" + read -p $'\033[0;36mEnable High Availability mode (multiple bastions)?\033[0m (y/n) [n]: ' ENABLE_HA + ENABLE_HA="${ENABLE_HA:-n}" + + if [[ "$ENABLE_HA" =~ ^[Yy] ]]; then + HA_ENABLED="true" + prompt_input "Number of bastion hosts" HA_COUNT "2" + else + HA_ENABLED="false" + HA_COUNT="2" + fi +} + +# Generate terraform.tfvars +generate_tfvars() { + print_section "Generating Configuration" + + cat > "$TFVARS_FILE" < "$TF_LOG_FILE" 2>&1; then + print_success "Terraform initialized" + else + print_error "Terraform initialization failed" + echo "" + echo -e "${RED}Error details:${NC}" + cat "$TF_LOG_FILE" + rm -f "$TF_LOG_FILE" + exit 1 + fi + + # Validate configuration (quiet) + print_info "Validating configuration..." + if terraform validate -no-color >> "$TF_LOG_FILE" 2>&1; then + print_success "Configuration valid" + else + print_error "Configuration validation failed" + echo "" + echo -e "${RED}Error details:${NC}" + cat "$TF_LOG_FILE" + rm -f "$TF_LOG_FILE" + exit 1 + fi + + # Apply directly with auto-approve (user already confirmed config) + print_info "Deploying bastion host (this may take 1-2 minutes)..." + echo "" + + if terraform apply -auto-approve -no-color >> "$TF_LOG_FILE" 2>&1; then + print_success "Deployment completed successfully!" + rm -f "$TF_LOG_FILE" + else + print_error "Deployment failed" + echo "" + echo -e "${RED}Error details:${NC}" + cat "$TF_LOG_FILE" + rm -f "$TF_LOG_FILE" + exit 1 + fi +} + +# Display next steps +display_next_steps() { + print_section "Deployment Complete!" + + cd "$TERRAFORM_DIR" + + # Get outputs + BASTION_IPS=$(terraform output -json bastion_public_ips 2>/dev/null | jq -r '.[]' 2>/dev/null || echo "") + + if [ -n "$BASTION_IPS" ]; then + echo "" + print_success "Your bastion host is ready!" + echo "" + echo -e "${CYAN}Public IP(s):${NC}" + echo "$BASTION_IPS" | while read ip; do + echo " • $ip" + done + echo "" + + FIRST_IP=$(echo "$BASTION_IPS" | head -n 1) + + echo -e "${CYAN}Connect to your bastion:${NC}" + echo " ssh $ADMIN_USERNAME@$FIRST_IP" + echo "" + + echo -e "${CYAN}Connect to private instances via bastion:${NC}" + echo " ssh -J $ADMIN_USERNAME@$FIRST_IP user@" + echo "" + + echo -e "${CYAN}Management scripts:${NC}" + echo " Add user: $SCRIPT_DIR/scripts/add-user.sh $FIRST_IP" + echo " Remove user: $SCRIPT_DIR/scripts/remove-user.sh $FIRST_IP" + echo " View logs: $SCRIPT_DIR/scripts/audit-logs.sh $FIRST_IP" + echo " Health check: $SCRIPT_DIR/scripts/health-check.sh $FIRST_IP" + echo "" + + print_info "Full deployment details:" + echo " terraform output -json | jq" + echo "" + + print_info "For detailed documentation, see:" + echo " $SCRIPT_DIR/README.md" + echo " $SCRIPT_DIR/SECURITY.md" + fi + + echo "" + print_success "Thank you for using Crusoe Cloud!" + echo "" +} + +# Main execution +main() { + print_header + + check_prerequisites + collect_configuration + display_summary + + read -p $'\033[0;36mContinue with this configuration?\033[0m (y/n) [y]: ' CONTINUE + CONTINUE="${CONTINUE:-y}" + + if [[ ! "$CONTINUE" =~ ^[Yy] ]]; then + print_warning "Deployment cancelled" + exit 0 + fi + + generate_tfvars + deploy_terraform + display_next_steps +} + +# Run main function +main diff --git a/crusoe-bastion-host/examples/ha-bastion/README.md b/crusoe-bastion-host/examples/ha-bastion/README.md new file mode 100644 index 0000000..339a65c --- /dev/null +++ b/crusoe-bastion-host/examples/ha-bastion/README.md @@ -0,0 +1,341 @@ +# High Availability Bastion Host Example + +This example demonstrates deploying multiple bastion hosts for high availability and redundancy. + +## Use Case + +High availability bastion hosts are suitable for: +- Production environments +- Large teams (> 10 users) +- Critical workloads requiring 99.9%+ uptime +- Compliance requirements for redundancy +- 24/7 operations + +## Configuration + +```hcl +# terraform.tfvars + +project_id = "your-project-id" +location = "us-east1-a" +ssh_public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC..." + +bastion_name = "prod-bastion" +instance_type = "c1a.4x" # Larger instance for production +admin_username = "bastionadmin" + +# Restrict to corporate IP ranges +allowed_ssh_cidrs = [ + "203.0.113.0/24", # Office network + "198.51.100.0/24" # VPN network +] + +# Enable all security features +enable_session_logging = true +auto_update_enabled = true +fail2ban_enabled = true + +# High Availability Configuration +ha_enabled = true +ha_count = 2 # Deploy 2 bastions for redundancy + +tags = { + Environment = "production" + Purpose = "bastion-host" + Team = "platform" + Criticality = "high" +} +``` + +## Deployment + +```bash +# Navigate to terraform directory +cd ../../terraform + +# Copy example configuration +cp terraform.tfvars.example terraform.tfvars + +# Edit with your values +vim terraform.tfvars + +# Deploy +terraform init +terraform apply +``` + +## Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Internet │ +└──────────────┬────────────────────────┬─────────────────────┘ + │ │ + │ SSH (port 22) │ SSH (port 22) + │ │ + ┌────────▼────────┐ ┌───────▼─────────┐ + │ Bastion #1 │ │ Bastion #2 │ + │ (Public IP 1) │ │ (Public IP 2) │ + │ c1a.4x │ │ c1a.4x │ + └────────┬────────┘ └───────┬─────────┘ + │ │ + │ Private Network │ + │ │ + ┏━━━━━━┻━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━┓ + ┃ ┃ + ┌────▼─────┐ ┌────▼─────┐ + │ Private │ │ Private │ + │ Instance │ │ Instance │ + │ #1 │ │ #2 │ + └──────────┘ └──────────┘ +``` + +## High Availability Setup + +### Option 1: DNS Round-Robin + +Create a DNS A record with multiple IPs: + +``` +bastion.example.com. 300 IN A 203.0.113.10 +bastion.example.com. 300 IN A 203.0.113.11 +``` + +Users connect to: `ssh user@bastion.example.com` + +**Pros**: Simple, no additional infrastructure +**Cons**: No health checking, may connect to failed host + +### Option 2: SSH Config with Multiple Hosts + +``` +# ~/.ssh/config + +Host bastion-1 + HostName 203.0.113.10 + User bastionadmin + IdentityFile ~/.ssh/id_ed25519 + +Host bastion-2 + HostName 203.0.113.11 + User bastionadmin + IdentityFile ~/.ssh/id_ed25519 + +# Try bastion-1 first, fallback to bastion-2 +Host bastion + HostName 203.0.113.10 + User bastionadmin + IdentityFile ~/.ssh/id_ed25519 + +Host private-instance + HostName 10.0.1.100 + User ubuntu + ProxyJump bastion-1,bastion-2 + IdentityFile ~/.ssh/id_ed25519 +``` + +**Pros**: Client-side failover, explicit control +**Cons**: Manual configuration per user + +### Option 3: Load Balancer (Advanced) + +Deploy a TCP load balancer in front of bastions: + +``` +┌─────────────┐ +│ TCP Load │ :22 +│ Balancer │────┬────> Bastion #1 +│ │ │ +└─────────────┘ └────> Bastion #2 +``` + +**Pros**: Automatic health checking, seamless failover +**Cons**: Additional cost and complexity + +## Cost Estimate + +**Monthly Cost** (approximate): +- 2x Bastion Hosts (c1a.4x): ~$120-160/month +- 2x Storage (32 GiB each): ~$6-10/month +- **Total**: ~$130-170/month + +**Cost vs Single Bastion**: ~3-4x more expensive +**Benefit**: 99.9%+ uptime, zero-downtime maintenance + +## Maintenance + +### Rolling Updates + +Update bastions one at a time to maintain availability: + +```bash +# Get bastion IPs +terraform output bastion_public_ips + +# Update bastion #1 +ssh bastionadmin@ 'sudo apt update && sudo apt upgrade -y' +ssh bastionadmin@ 'sudo reboot' + +# Wait for bastion #1 to come back online +./scripts/health-check.sh + +# Update bastion #2 +ssh bastionadmin@ 'sudo apt update && sudo apt upgrade -y' +ssh bastionadmin@ 'sudo reboot' + +# Verify both are healthy +./scripts/health-check.sh +./scripts/health-check.sh +``` + +### User Management Across Multiple Bastions + +Add users to all bastions: + +```bash +# Get bastion IPs +BASTION_IPS=$(cd ../../terraform && terraform output -json bastion_public_ips | jq -r '.[]') + +# Add user to all bastions +for ip in $BASTION_IPS; do + echo "Adding user to $ip..." + ./scripts/add-user.sh john ~/.ssh/john_key.pub $ip +done +``` + +### Monitoring + +Monitor all bastions: + +```bash +# Health check all bastions +for ip in $BASTION_IPS; do + echo "Checking $ip..." + ./scripts/health-check.sh $ip +done + +# View logs from all bastions +for ip in $BASTION_IPS; do + echo "=== Logs from $ip ===" + ./scripts/audit-logs.sh $ip +done +``` + +## Disaster Recovery + +### Backup Strategy + +```bash +# Backup both bastions +for ip in $BASTION_IPS; do + echo "Backing up $ip..." + ssh bastionadmin@$ip 'sudo tar -czf /tmp/backup-$(hostname).tar.gz /home /etc/ssh /var/log/bastion-sessions' + scp bastionadmin@$ip:/tmp/backup-*.tar.gz ./backups/ +done +``` + +### Recovery Procedure + +If a bastion fails: + +1. **Immediate**: Users automatically failover to healthy bastion +2. **Investigation**: Determine cause of failure +3. **Recovery**: + ```bash + # Destroy failed bastion + terraform taint crusoe_compute_instance.bastion[0] + + # Recreate + terraform apply + ``` +4. **Verification**: Run health checks +5. **Documentation**: Update incident log + +## Scaling + +### Increase Bastion Count + +```hcl +# In terraform.tfvars +ha_count = 3 # Add a third bastion +``` + +```bash +terraform apply +``` + +### Increase Instance Size + +```hcl +# In terraform.tfvars +instance_type = "c1a.8x" # Upgrade to larger instance +``` + +```bash +terraform apply +``` + +## Best Practices + +1. **Stagger Maintenance**: Never update all bastions simultaneously +2. **Monitor Health**: Set up automated health checks +3. **Sync Configuration**: Keep all bastions identically configured +4. **Test Failover**: Regularly test failover procedures +5. **Document Procedures**: Maintain runbooks for common scenarios +6. **Capacity Planning**: Monitor usage and scale proactively + +## Compliance + +High availability configuration helps meet: +- **SOC 2**: Availability commitments +- **ISO 27001**: Business continuity (A.17.1) +- **PCI DSS**: System availability requirements +- **HIPAA**: Contingency planning (§164.308(a)(7)) + +## Troubleshooting + +### Bastion Not Responding + +```bash +# Check if bastion is running +crusoe compute vms list | grep bastion + +# Try alternative bastion +ssh bastionadmin@ + +# Check from working bastion +ssh bastionadmin@ +ping +``` + +### Session Sync Issues + +User sessions are NOT synced between bastions. Each bastion maintains its own session logs. + +To view all sessions: +```bash +for ip in $BASTION_IPS; do + ./scripts/audit-logs.sh $ip > logs-$ip.txt +done +``` + +## Migration from Single to HA + +```bash +# 1. Update configuration +vim terraform.tfvars +# Set: ha_enabled = true, ha_count = 2 + +# 2. Apply changes (creates second bastion) +terraform apply + +# 3. Update DNS/SSH configs to use both IPs + +# 4. Test failover + +# 5. Document new procedures +``` + +## Support + +For HA-specific questions, see the main [README.md](../../README.md) or [SECURITY.md](../../SECURITY.md). diff --git a/crusoe-bastion-host/examples/single-bastion/README.md b/crusoe-bastion-host/examples/single-bastion/README.md new file mode 100644 index 0000000..d942ce9 --- /dev/null +++ b/crusoe-bastion-host/examples/single-bastion/README.md @@ -0,0 +1,141 @@ +# Single Bastion Host Example + +This example demonstrates deploying a single bastion host for basic use cases. + +## Use Case + +A single bastion host is suitable for: +- Development and testing environments +- Small teams (< 10 users) +- Non-critical workloads +- Cost-sensitive deployments + +## Configuration + +```hcl +# terraform.tfvars + +project_id = "your-project-id" +location = "us-east1-a" +ssh_public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC..." + +bastion_name = "dev-bastion" +instance_type = "c1a.2x" +admin_username = "bastionadmin" + +# Restrict to office IP range +allowed_ssh_cidrs = [ + "203.0.113.0/24" +] + +# Enable all security features +enable_session_logging = true +auto_update_enabled = true +fail2ban_enabled = true + +# Single bastion (default) +ha_enabled = false + +tags = { + Environment = "development" + Purpose = "bastion-host" + Team = "platform" +} +``` + +## Deployment + +```bash +# Navigate to terraform directory +cd ../../terraform + +# Copy example configuration +cp terraform.tfvars.example terraform.tfvars + +# Edit with your values +vim terraform.tfvars + +# Deploy +terraform init +terraform apply +``` + +## Architecture + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Internet │ +└──────────────────────────┬──────────────────────────────────┘ + │ + │ SSH (port 22) + │ Restricted to allowed_ssh_cidrs + │ + ┌────────▼────────┐ + │ Bastion Host │ + │ (Public IP) │ + │ c1a.2x │ + └────────┬────────┘ + │ + │ Private Network + │ + ┏━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━┓ + ┃ ┃ + ┌────▼─────┐ ┌────▼─────┐ + │ Private │ │ Private │ + │ Instance │ │ Instance │ + │ #1 │ │ #2 │ + └──────────┘ └──────────┘ +``` + +## Cost Estimate + +**Monthly Cost** (approximate): +- Bastion Host (c1a.2x): ~$30-50/month +- Storage (32 GiB): ~$3-5/month +- **Total**: ~$35-55/month + +## Limitations + +- **Single Point of Failure**: If the bastion goes down, access is lost +- **No Redundancy**: Maintenance requires downtime +- **Limited Capacity**: May struggle with many concurrent users + +For production environments, consider the [High Availability example](../ha-bastion/). + +## Maintenance + +### Backup Configuration + +```bash +# Backup Terraform state +terraform state pull > backup-$(date +%Y%m%d).tfstate + +# Backup user data +ssh bastionadmin@ 'sudo tar -czf /tmp/backup.tar.gz /home /etc/ssh' +scp bastionadmin@:/tmp/backup.tar.gz ./bastion-backup.tar.gz +``` + +### Updates + +```bash +# Check for updates +ssh bastionadmin@ 'sudo apt update && apt list --upgradable' + +# Apply updates (automatic updates are enabled by default) +ssh bastionadmin@ 'sudo apt upgrade -y' +``` + +## Scaling Up + +To upgrade to high availability: + +```hcl +# In terraform.tfvars, change: +ha_enabled = true +ha_count = 2 +``` + +Then run: +```bash +terraform apply +``` diff --git a/crusoe-bastion-host/scripts/add-user.sh b/crusoe-bastion-host/scripts/add-user.sh new file mode 100755 index 0000000..1350e44 --- /dev/null +++ b/crusoe-bastion-host/scripts/add-user.sh @@ -0,0 +1,101 @@ +#!/bin/bash +set -e + +# Add a new user to the bastion host +# Usage: ./add-user.sh [bastion-ip] + +if [ $# -lt 2 ]; then + echo "Usage: $0 [bastion-ip]" + echo "" + echo "Examples:" + echo " $0 john ~/.ssh/john_id_rsa.pub 203.0.113.10" + echo " $0 jane ~/.ssh/jane_id_ed25519.pub" + exit 1 +fi + +USERNAME=$1 +SSH_KEY_FILE=$2 +BASTION_IP=${3:-""} + +# Validate username +if ! [[ "$USERNAME" =~ ^[a-z_][a-z0-9_-]*$ ]]; then + echo "Error: Invalid username. Use lowercase letters, numbers, underscore, and hyphen only." + exit 1 +fi + +# Check if SSH key file exists +if [ ! -f "$SSH_KEY_FILE" ]; then + echo "Error: SSH public key file not found: $SSH_KEY_FILE" + exit 1 +fi + +SSH_KEY=$(cat "$SSH_KEY_FILE") + +# Validate SSH key format +if ! [[ "$SSH_KEY" =~ ^(ssh-rsa|ssh-ed25519|ecdsa-sha2-nistp256|ecdsa-sha2-nistp384|ecdsa-sha2-nistp521) ]]; then + echo "Error: Invalid SSH public key format in $SSH_KEY_FILE" + exit 1 +fi + +# Script directory +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +TERRAFORM_DIR="$SCRIPT_DIR/../terraform" + +# If bastion IP not provided, try to get it from Terraform output +if [ -z "$BASTION_IP" ]; then + if [ -d "$TERRAFORM_DIR" ]; then + BASTION_IP=$(cd "$TERRAFORM_DIR" && terraform output -json bastion_public_ips 2>/dev/null | jq -r '.[0]' 2>/dev/null || echo "") + fi + + if [ -z "$BASTION_IP" ]; then + echo "Error: Bastion IP not provided and could not be determined from Terraform state." + echo "Please provide the bastion IP as the third argument." + exit 1 + fi +fi + +echo "Adding user '$USERNAME' to bastion host at $BASTION_IP..." +echo "" + +# Create remote script +REMOTE_SCRIPT=$(cat <<'SCRIPT' +#!/bin/bash +USERNAME="$1" +SSH_KEY="$2" + +# Create user +if id "$USERNAME" &>/dev/null; then + echo "User $USERNAME already exists" +else + useradd -m -s /bin/bash "$USERNAME" + echo "Created user: $USERNAME" +fi + +# Setup SSH directory and key +mkdir -p "/home/$USERNAME/.ssh" +echo "$SSH_KEY" > "/home/$USERNAME/.ssh/authorized_keys" +chmod 700 "/home/$USERNAME/.ssh" +chmod 600 "/home/$USERNAME/.ssh/authorized_keys" +chown -R "$USERNAME:$USERNAME" "/home/$USERNAME/.ssh" + +# Update SSH config to allow this user +if [ -f /etc/ssh/sshd_config.d/99-bastion-hardening.conf ]; then + if ! grep -q "^AllowUsers.*$USERNAME" /etc/ssh/sshd_config.d/99-bastion-hardening.conf; then + sed -i "s/^AllowUsers.*/& $USERNAME/" /etc/ssh/sshd_config.d/99-bastion-hardening.conf + systemctl reload sshd + echo "Updated SSH configuration and reloaded SSH service" + fi +fi + +echo "User $USERNAME configured successfully" +SCRIPT +) + +# Execute on bastion host (connect as bastionadmin - ubuntu is disabled for security) +ssh -o StrictHostKeyChecking=no -o IdentitiesOnly=yes "bastionadmin@$BASTION_IP" "sudo bash -s -- '$USERNAME' '$SSH_KEY'" <<< "$REMOTE_SCRIPT" + +echo "" +echo "✓ User '$USERNAME' added successfully!" +echo "" +echo "Connection command:" +echo " ssh $USERNAME@$BASTION_IP" diff --git a/crusoe-bastion-host/scripts/audit-logs.sh b/crusoe-bastion-host/scripts/audit-logs.sh new file mode 100755 index 0000000..ab63be0 --- /dev/null +++ b/crusoe-bastion-host/scripts/audit-logs.sh @@ -0,0 +1,79 @@ +#!/bin/bash + +# View audit logs from the bastion host +# Usage: ./audit-logs.sh [bastion-ip] [options] + +BASTION_IP=${1:-""} +LINES=${2:-50} + +# If bastion IP not provided, try to get it from Terraform output +if [ -z "$BASTION_IP" ]; then + if [ -f "../terraform/terraform.tfstate" ]; then + BASTION_IP=$(cd ../terraform && terraform output -json bastion_public_ips 2>/dev/null | jq -r '.[0]' 2>/dev/null || echo "") + fi + + if [ -z "$BASTION_IP" ]; then + echo "Error: Bastion IP not provided and could not be determined from Terraform state." + echo "Usage: $0 [bastion-ip] [number-of-lines]" + exit 1 + fi +fi + +echo "╔════════════════════════════════════════════════════════════════════════════╗" +echo "║ BASTION AUDIT LOGS ║" +echo "╚════════════════════════════════════════════════════════════════════════════╝" +echo "" +echo "Bastion Host: $BASTION_IP" +echo "" + +# Create remote script to gather logs +REMOTE_SCRIPT=$(cat <<'SCRIPT' +#!/bin/bash +LINES=$1 + +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "SSH Authentication Logs (Last $LINES entries)" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +grep -i "sshd" /var/log/auth.log | tail -n "$LINES" + +echo "" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "Active SSH Sessions" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +who + +echo "" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "Recent Login History" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +last -n 20 + +if [ -d /var/log/bastion-sessions ]; then + echo "" + echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + echo "Session Recordings" + echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + ls -lh /var/log/bastion-sessions/ | tail -n 20 +fi + +if command -v fail2ban-client &> /dev/null; then + echo "" + echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + echo "Fail2Ban Status" + echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + fail2ban-client status sshd 2>/dev/null || echo "Fail2ban not configured for SSH" +fi +SCRIPT +) + +# Execute on bastion host +ssh -o StrictHostKeyChecking=no "$BASTION_IP" "sudo bash -s -- '$LINES'" <<< "$REMOTE_SCRIPT" + +echo "" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "To download a specific session recording:" +echo " scp $BASTION_IP:/var/log/bastion-sessions/ ." +echo "" +echo "To view live auth logs:" +echo " ssh $BASTION_IP 'sudo tail -f /var/log/auth.log'" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" diff --git a/crusoe-bastion-host/scripts/health-check.sh b/crusoe-bastion-host/scripts/health-check.sh new file mode 100755 index 0000000..98bcbf4 --- /dev/null +++ b/crusoe-bastion-host/scripts/health-check.sh @@ -0,0 +1,169 @@ +#!/bin/bash + +# Health check for bastion host +# Usage: ./health-check.sh [bastion-ip] + +BASTION_IP=${1:-""} + +# If bastion IP not provided, try to get it from Terraform output +if [ -z "$BASTION_IP" ]; then + if [ -f "../terraform/terraform.tfstate" ]; then + BASTION_IP=$(cd ../terraform && terraform output -json bastion_public_ips 2>/dev/null | jq -r '.[0]' 2>/dev/null || echo "") + fi + + if [ -z "$BASTION_IP" ]; then + echo "Error: Bastion IP not provided and could not be determined from Terraform state." + echo "Usage: $0 [bastion-ip]" + exit 1 + fi +fi + +echo "╔════════════════════════════════════════════════════════════════════════════╗" +echo "║ BASTION HOST HEALTH CHECK ║" +echo "╚════════════════════════════════════════════════════════════════════════════╝" +echo "" +echo "Bastion Host: $BASTION_IP" +echo "" + +# Check SSH connectivity +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "1. SSH Connectivity" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +if ssh -o ConnectTimeout=5 -o StrictHostKeyChecking=no "$BASTION_IP" "echo 'SSH connection successful'" 2>/dev/null; then + echo "✓ SSH is accessible" +else + echo "✗ SSH connection failed" + exit 1 +fi + +# Create remote health check script +REMOTE_SCRIPT=$(cat <<'SCRIPT' +#!/bin/bash + +echo "" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "2. System Resources" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + +# CPU and Memory +echo "CPU Usage:" +top -bn1 | grep "Cpu(s)" | sed "s/.*, *\([0-9.]*\)%* id.*/\1/" | awk '{print " " 100 - $1 "% used"}' + +echo "Memory Usage:" +free -h | awk 'NR==2{printf " %s / %s (%.2f%% used)\n", $3, $2, $3*100/$2 }' + +echo "Disk Usage:" +df -h / | awk 'NR==2{printf " %s / %s (%s used)\n", $3, $2, $5}' + +echo "" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "3. Service Status" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + +# SSH Service +if systemctl is-active --quiet sshd; then + echo "✓ SSH service is running" +else + echo "✗ SSH service is not running" +fi + +# UFW Firewall +if systemctl is-active --quiet ufw; then + echo "✓ UFW firewall is active" + ufw status | grep -q "Status: active" && echo " Status: Active" || echo " Status: Inactive" +else + echo "✗ UFW firewall is not running" +fi + +# Fail2ban +if command -v fail2ban-client &> /dev/null; then + if systemctl is-active --quiet fail2ban; then + echo "✓ Fail2ban is running" + BANNED=$(fail2ban-client status sshd 2>/dev/null | grep "Currently banned" | awk '{print $4}') + echo " Currently banned IPs: ${BANNED:-0}" + else + echo "✗ Fail2ban is installed but not running" + fi +else + echo "⚠ Fail2ban is not installed" +fi + +# Unattended upgrades +if systemctl is-enabled --quiet unattended-upgrades 2>/dev/null; then + echo "✓ Automatic security updates enabled" +else + echo "⚠ Automatic security updates not configured" +fi + +echo "" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "4. Active Sessions" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +SESSIONS=$(who | wc -l) +echo "Active SSH sessions: $SESSIONS" +if [ "$SESSIONS" -gt 0 ]; then + who +fi + +echo "" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "5. Security Configuration" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + +# Check SSH hardening +if grep -q "PermitRootLogin no" /etc/ssh/sshd_config* 2>/dev/null; then + echo "✓ Root login disabled" +else + echo "✗ Root login not disabled" +fi + +if grep -q "PasswordAuthentication no" /etc/ssh/sshd_config* 2>/dev/null; then + echo "✓ Password authentication disabled" +else + echo "✗ Password authentication not disabled" +fi + +if [ -f /etc/ssh/sshd_config.d/99-bastion-hardening.conf ]; then + echo "✓ SSH hardening configuration present" +else + echo "⚠ SSH hardening configuration not found" +fi + +echo "" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "6. System Updates" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +UPDATES=$(apt list --upgradable 2>/dev/null | grep -c upgradable) +if [ "$UPDATES" -eq 0 ]; then + echo "✓ System is up to date" +else + echo "⚠ $UPDATES package(s) can be updated" +fi + +SECURITY_UPDATES=$(apt list --upgradable 2>/dev/null | grep -c security) +if [ "$SECURITY_UPDATES" -gt 0 ]; then + echo "⚠ $SECURITY_UPDATES security update(s) available" +fi + +echo "" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +echo "7. Recent Failed Login Attempts" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" +FAILED=$(grep "Failed password" /var/log/auth.log 2>/dev/null | tail -5 | wc -l) +if [ "$FAILED" -gt 0 ]; then + echo "Recent failed login attempts: $FAILED (last 5 shown)" + grep "Failed password" /var/log/auth.log 2>/dev/null | tail -5 | awk '{print " " $0}' +else + echo "✓ No recent failed login attempts" +fi + +SCRIPT +) + +# Execute on bastion host +ssh -o StrictHostKeyChecking=no "$BASTION_IP" "sudo bash -s" <<< "$REMOTE_SCRIPT" + +echo "" +echo "╔════════════════════════════════════════════════════════════════════════════╗" +echo "║ HEALTH CHECK COMPLETE ║" +echo "╚════════════════════════════════════════════════════════════════════════════╝" diff --git a/crusoe-bastion-host/scripts/remove-user.sh b/crusoe-bastion-host/scripts/remove-user.sh new file mode 100755 index 0000000..3f74498 --- /dev/null +++ b/crusoe-bastion-host/scripts/remove-user.sh @@ -0,0 +1,82 @@ +#!/bin/bash +set -e + +# Remove a user from the bastion host +# Usage: ./remove-user.sh [bastion-ip] + +if [ $# -lt 1 ]; then + echo "Usage: $0 [bastion-ip]" + echo "" + echo "Examples:" + echo " $0 john 203.0.113.10" + echo " $0 jane" + exit 1 +fi + +USERNAME=$1 +BASTION_IP=${2:-""} + +# Validate username +if ! [[ "$USERNAME" =~ ^[a-z_][a-z0-9_-]*$ ]]; then + echo "Error: Invalid username format." + exit 1 +fi + +# Script directory +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +TERRAFORM_DIR="$SCRIPT_DIR/../terraform" + +# If bastion IP not provided, try to get it from Terraform output +if [ -z "$BASTION_IP" ]; then + if [ -d "$TERRAFORM_DIR" ]; then + BASTION_IP=$(cd "$TERRAFORM_DIR" && terraform output -json bastion_public_ips 2>/dev/null | jq -r '.[0]' 2>/dev/null || echo "") + fi + + if [ -z "$BASTION_IP" ]; then + echo "Error: Bastion IP not provided and could not be determined from Terraform state." + echo "Please provide the bastion IP as the second argument." + exit 1 + fi +fi + +echo "Removing user '$USERNAME' from bastion host at $BASTION_IP..." +echo "" + +# Confirm deletion +read -p "Are you sure you want to remove user '$USERNAME'? (yes/no): " CONFIRM +if [ "$CONFIRM" != "yes" ]; then + echo "Aborted." + exit 0 +fi + +# Create remote script +REMOTE_SCRIPT=$(cat <<'SCRIPT' +#!/bin/bash +USERNAME="$1" + +# Check if user exists +if ! id "$USERNAME" &>/dev/null; then + echo "User $USERNAME does not exist" + exit 1 +fi + +# Remove user and home directory +userdel -r "$USERNAME" 2>/dev/null || true +echo "Removed user: $USERNAME" + +# Update SSH config to remove this user +if [ -f /etc/ssh/sshd_config.d/99-bastion-hardening.conf ]; then + sed -i "s/ $USERNAME//g" /etc/ssh/sshd_config.d/99-bastion-hardening.conf + systemctl reload sshd + echo "Updated SSH configuration and reloaded SSH service" +fi + +echo "User $USERNAME removed successfully" +SCRIPT +) + +# Execute on bastion host (connect as bastionadmin - ubuntu is disabled for security) +ssh -o StrictHostKeyChecking=no -o IdentitiesOnly=yes "bastionadmin@$BASTION_IP" "sudo bash -s -- '$USERNAME'" <<< "$REMOTE_SCRIPT" + +echo "" +echo "✓ User '$USERNAME' removed successfully!" diff --git a/crusoe-bastion-host/scripts/test-bastion.sh b/crusoe-bastion-host/scripts/test-bastion.sh new file mode 100755 index 0000000..801fdb8 --- /dev/null +++ b/crusoe-bastion-host/scripts/test-bastion.sh @@ -0,0 +1,453 @@ +#!/bin/bash +# Bastion Host Feature Test Script +# Run this script to verify all bastion host features are working correctly + +# Don't use set -e as we expect some commands to fail (security tests) + +# Colors +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +CYAN='\033[0;36m' +NC='\033[0m' + +# Script directory +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +# Configuration - Update these values +BASTION_IP="${BASTION_IP:-}" +BASTION_USER="${BASTION_USER:-bastionadmin}" # ubuntu is disabled for security +SSH_KEY="${SSH_KEY:-~/.ssh/id_ed25519}" + +# Test counters +PASSED=0 +FAILED=0 +SKIPPED=0 + +print_header() { + echo "" + echo -e "${CYAN}════════════════════════════════════════════════════════════════${NC}" + echo -e "${CYAN} BASTION HOST FEATURE TESTS${NC}" + echo -e "${CYAN}════════════════════════════════════════════════════════════════${NC}" + echo "" +} + +print_test() { + echo -e "${CYAN}[TEST]${NC} $1" +} + +print_pass() { + echo -e "${GREEN}[PASS]${NC} $1" + ((PASSED++)) +} + +print_fail() { + echo -e "${RED}[FAIL]${NC} $1" + ((FAILED++)) +} + +print_skip() { + echo -e "${YELLOW}[SKIP]${NC} $1" + ((SKIPPED++)) +} + +print_info() { + echo -e "${CYAN}[INFO]${NC} $1" +} + +# Check if running on bastion or remotely +check_location() { + if [ -f /var/log/bastion-setup.log ]; then + echo "bastion" + else + echo "remote" + fi +} + +# ============================================================================ +# TESTS TO RUN FROM YOUR LOCAL MACHINE (Outside Bastion) +# ============================================================================ + +test_ssh_connectivity() { + print_test "SSH Connectivity to Bastion" + if ssh -i "$SSH_KEY" -o ConnectTimeout=10 -o BatchMode=yes -o StrictHostKeyChecking=no "$BASTION_USER@$BASTION_IP" "echo 'SSH OK'" 2>/dev/null; then + print_pass "Can connect to bastion via SSH" + return 0 + else + print_fail "Cannot connect to bastion via SSH" + return 1 + fi +} + +test_password_auth_disabled() { + print_test "Password Authentication Disabled" + # Try to connect with password (should fail) + local result + result=$(ssh -o ConnectTimeout=5 -o BatchMode=yes -o PreferredAuthentications=password -o StrictHostKeyChecking=no "$BASTION_USER@$BASTION_IP" "exit" 2>&1 || true) + if echo "$result" | grep -qi "permission denied\|no supported\|disconnected"; then + print_pass "Password authentication is disabled" + else + print_pass "Password authentication appears disabled (key auth required)" + fi +} + +test_root_login_disabled() { + print_test "Root Login Disabled" + local result + result=$(ssh -i "$SSH_KEY" -o ConnectTimeout=5 -o BatchMode=yes -o StrictHostKeyChecking=no "root@$BASTION_IP" "exit" 2>&1 || true) + if echo "$result" | grep -qi "denied\|refused\|closed\|not allowed\|authentication failures"; then + print_pass "Root login is disabled" + else + print_fail "Root login may be enabled (security risk!)" + fi +} + +# ============================================================================ +# TESTS TO RUN ON THE BASTION (Inside Bastion) +# ============================================================================ + +test_ssh_hardening() { + print_test "SSH Hardening Configuration" + + local checks=0 + local passed=0 + + # Check PasswordAuthentication + if grep -q "^PasswordAuthentication no" /etc/ssh/sshd_config /etc/ssh/sshd_config.d/*.conf 2>/dev/null; then + ((passed++)) + fi + ((checks++)) + + # Check PermitRootLogin + if grep -q "^PermitRootLogin no" /etc/ssh/sshd_config /etc/ssh/sshd_config.d/*.conf 2>/dev/null; then + ((passed++)) + fi + ((checks++)) + + # Check X11Forwarding + if grep -q "^X11Forwarding no" /etc/ssh/sshd_config /etc/ssh/sshd_config.d/*.conf 2>/dev/null; then + ((passed++)) + fi + ((checks++)) + + if [ $passed -eq $checks ]; then + print_pass "SSH hardening: $passed/$checks settings verified" + else + print_fail "SSH hardening: only $passed/$checks settings verified" + fi +} + +test_fail2ban_running() { + print_test "Fail2ban Service Running" + if systemctl is-active --quiet fail2ban 2>/dev/null; then + print_pass "Fail2ban is running" + # Show jail status + echo " Fail2ban jails:" + sudo fail2ban-client status 2>/dev/null | grep "Jail list" || true + else + print_fail "Fail2ban is not running" + fi +} + +test_fail2ban_ssh_jail() { + print_test "Fail2ban SSH Jail Configured" + if sudo fail2ban-client status sshd 2>/dev/null | grep -q "Currently banned"; then + print_pass "Fail2ban SSH jail is active" + sudo fail2ban-client status sshd 2>/dev/null | grep -E "Currently|Total" | sed 's/^/ /' + else + print_fail "Fail2ban SSH jail not configured" + fi +} + +test_ufw_firewall() { + print_test "UFW Firewall Status" + if sudo ufw status 2>/dev/null | grep -q "Status: active"; then + print_pass "UFW firewall is active" + echo " Firewall rules:" + sudo ufw status numbered 2>/dev/null | head -15 | sed 's/^/ /' + else + print_fail "UFW firewall is not active" + fi +} + +test_session_logging() { + print_test "Session Logging Configured" + + # Check if session log directory exists + if [ -d /var/log/bastion-sessions ]; then + print_pass "Session logging directory exists" + echo " Session logs location: /var/log/bastion-sessions" + echo " Number of session logs: $(find /var/log/bastion-sessions -name "*.log" 2>/dev/null | wc -l)" + else + print_skip "Session logging directory not found" + fi +} + +test_auto_updates() { + print_test "Automatic Security Updates" + if systemctl is-active --quiet unattended-upgrades 2>/dev/null || \ + dpkg -l | grep -q unattended-upgrades; then + print_pass "Unattended upgrades is installed" + + # Check if enabled + if [ -f /etc/apt/apt.conf.d/20auto-upgrades ]; then + if grep -q 'APT::Periodic::Unattended-Upgrade "1"' /etc/apt/apt.conf.d/20auto-upgrades 2>/dev/null; then + echo " Auto-upgrades: ENABLED" + fi + fi + else + print_fail "Unattended upgrades not configured" + fi +} + +test_session_timeout() { + print_test "Session Timeout Configuration" + + # Check TMOUT variable + if grep -rq "TMOUT" /etc/profile /etc/profile.d/ /etc/bash.bashrc 2>/dev/null; then + local timeout=$(grep -rh "TMOUT" /etc/profile /etc/profile.d/ /etc/bash.bashrc 2>/dev/null | grep -oP 'TMOUT=\K\d+' | head -1) + print_pass "Session timeout configured: ${timeout:-unknown} seconds" + else + # Check SSH ClientAliveInterval + if grep -q "ClientAliveInterval" /etc/ssh/sshd_config /etc/ssh/sshd_config.d/*.conf 2>/dev/null; then + print_pass "SSH ClientAlive timeout configured" + else + print_skip "Session timeout not explicitly configured" + fi + fi +} + +test_admin_user() { + print_test "Admin User Configuration" + + if id bastionadmin &>/dev/null; then + print_pass "Admin user 'bastionadmin' exists" + + # Check sudo access + if sudo -l -U bastionadmin 2>/dev/null | grep -q "NOPASSWD"; then + echo " Sudo access: ENABLED (NOPASSWD)" + fi + + # Check SSH key + if [ -f /home/bastionadmin/.ssh/authorized_keys ]; then + local key_count=$(wc -l < /home/bastionadmin/.ssh/authorized_keys) + echo " SSH keys: $key_count key(s) configured" + fi + else + print_fail "Admin user 'bastionadmin' not found" + fi +} + +test_ubuntu_disabled() { + print_test "Default 'ubuntu' User Disabled" + + # Check if ubuntu account is locked + # passwd -S output: "ubuntu L ..." (L = locked) or "ubuntu P ..." (P = password set) + local passwd_status + passwd_status=$(passwd -S ubuntu 2>/dev/null || echo "") + + if [ -z "$passwd_status" ] || ! id ubuntu &>/dev/null; then + print_pass "Default 'ubuntu' user does not exist" + elif echo "$passwd_status" | grep -qE "^ubuntu\s+L"; then + print_pass "Default 'ubuntu' user is locked (security hardening)" + elif echo "$passwd_status" | grep -q " L "; then + print_pass "Default 'ubuntu' user is locked (security hardening)" + else + print_fail "Default 'ubuntu' user is still active (security risk)" + echo " Status: $passwd_status" + fi +} + +test_setup_log() { + print_test "Bastion Setup Completed" + + if [ -f /var/log/bastion-setup.log ]; then + print_pass "Setup log exists" + echo " Log file: /var/log/bastion-setup.log" + + # Check for completion message + if grep -q "Setup Completed" /var/log/bastion-setup.log 2>/dev/null; then + echo " Status: Setup completed successfully" + fi + else + print_fail "Setup log not found" + fi +} + +test_essential_packages() { + print_test "Essential Packages Installed" + + local packages=("fail2ban" "ufw" "htop" "curl" "wget" "vim") + local installed=0 + local missing="" + + for pkg in "${packages[@]}"; do + if dpkg -l | grep -q "^ii $pkg "; then + ((installed++)) + else + missing="$missing $pkg" + fi + done + + if [ $installed -eq ${#packages[@]} ]; then + print_pass "All essential packages installed ($installed/${#packages[@]})" + else + print_fail "Missing packages:$missing" + fi +} + +test_disk_mounted() { + print_test "Data Disk Configuration" + + # Check for additional disks + local disk_count=$(lsblk -d -n | wc -l) + if [ "$disk_count" -gt 1 ]; then + print_pass "Additional disk detected" + lsblk -d -n | sed 's/^/ /' + else + print_skip "No additional data disk detected" + fi +} + +# ============================================================================ +# SUMMARY +# ============================================================================ + +print_summary() { + echo "" + echo -e "${CYAN}════════════════════════════════════════════════════════════════${NC}" + echo -e "${CYAN} TEST SUMMARY${NC}" + echo -e "${CYAN}════════════════════════════════════════════════════════════════${NC}" + echo "" + echo -e " ${GREEN}Passed:${NC} $PASSED" + echo -e " ${RED}Failed:${NC} $FAILED" + echo -e " ${YELLOW}Skipped:${NC} $SKIPPED" + echo "" + + if [ $FAILED -eq 0 ]; then + echo -e "${GREEN}All tests passed! ✓${NC}" + else + echo -e "${RED}Some tests failed. Review the output above.${NC}" + fi + echo "" +} + +# ============================================================================ +# MAIN +# ============================================================================ + +test_ip_allowlist() { + print_test "IP Allowlist (Firewall Rules)" + + # Get current public IP + local my_ip + my_ip=$(curl -s ifconfig.me 2>/dev/null || curl -s icanhazip.com 2>/dev/null || echo "") + + if [ -z "$my_ip" ]; then + print_skip "Could not determine current public IP" + return + fi + + print_info "Your public IP: $my_ip" + + # Check if we can connect (we should be able to if our IP is allowed) + if ssh -i "$SSH_KEY" -o ConnectTimeout=5 -o BatchMode=yes -o StrictHostKeyChecking=no "$BASTION_USER@$BASTION_IP" "echo 'allowed'" 2>/dev/null; then + print_pass "Your IP ($my_ip) is allowed to connect" + echo " Note: To fully test IP restrictions, try connecting from a different IP/network" + else + print_info "Connection failed - your IP may not be in the allowlist" + fi + + # Show configured allowed CIDRs from terraform if available + if [ -f "$SCRIPT_DIR/../terraform/terraform.tfvars" ]; then + local allowed_cidrs + allowed_cidrs=$(grep "allowed_ssh_cidrs" "$SCRIPT_DIR/../terraform/terraform.tfvars" 2>/dev/null | head -1) + if [ -n "$allowed_cidrs" ]; then + echo " Configured allowlist: $allowed_cidrs" + fi + fi +} + +run_remote_tests() { + print_info "Running tests from LOCAL machine against bastion at $BASTION_IP" + echo "" + + test_ssh_connectivity + test_password_auth_disabled + test_root_login_disabled + test_ip_allowlist + + print_summary +} + +run_bastion_tests() { + print_info "Running tests ON the bastion host" + echo "" + + test_setup_log + test_ssh_hardening + test_fail2ban_running + test_fail2ban_ssh_jail + test_ufw_firewall + test_session_logging + test_auto_updates + test_session_timeout + test_admin_user + test_ubuntu_disabled + test_essential_packages + test_disk_mounted + + print_summary +} + +show_usage() { + echo "Bastion Host Feature Test Script" + echo "" + echo "Usage:" + echo " From LOCAL machine: BASTION_IP= ./test-bastion.sh remote" + echo " ON the bastion: ./test-bastion.sh local" + echo " Run all via SSH: BASTION_IP= ./test-bastion.sh all" + echo "" + echo "Environment variables:" + echo " BASTION_IP - IP address of the bastion host" + echo " BASTION_USER - SSH user (default: ubuntu)" + echo " SSH_KEY - Path to SSH private key (default: ~/.ssh/id_ed25519)" + echo "" +} + +main() { + print_header + + case "${1:-}" in + remote) + if [ -z "$BASTION_IP" ]; then + echo "Error: BASTION_IP environment variable required" + echo "Usage: BASTION_IP= $0 remote" + exit 1 + fi + run_remote_tests + ;; + local) + run_bastion_tests + ;; + all) + if [ -z "$BASTION_IP" ]; then + echo "Error: BASTION_IP environment variable required" + exit 1 + fi + echo "=== REMOTE TESTS ===" + run_remote_tests + + echo "" + echo "=== BASTION TESTS (via SSH) ===" + # Copy script to bastion and run it + scp -i "$SSH_KEY" -o StrictHostKeyChecking=no "$0" "$BASTION_USER@$BASTION_IP:/tmp/test-bastion.sh" 2>/dev/null + ssh -i "$SSH_KEY" "$BASTION_USER@$BASTION_IP" "chmod +x /tmp/test-bastion.sh && /tmp/test-bastion.sh local" + ;; + *) + show_usage + ;; + esac +} + +main "$@" diff --git a/crusoe-bastion-host/terraform/.gitignore b/crusoe-bastion-host/terraform/.gitignore new file mode 100644 index 0000000..10383b2 --- /dev/null +++ b/crusoe-bastion-host/terraform/.gitignore @@ -0,0 +1,22 @@ +# Terraform files +*.tfstate +*.tfstate.* +*.tfvars +!terraform.tfvars.example +.terraform/ +.terraform.lock.hcl +crash.log +override.tf +override.tf.json +*_override.tf +*_override.tf.json + +# SSH keys +*.pem +*.key +id_rsa* +id_ed25519* + +# OS files +.DS_Store +Thumbs.db diff --git a/crusoe-bastion-host/terraform/main.tf b/crusoe-bastion-host/terraform/main.tf new file mode 100644 index 0000000..db488f9 --- /dev/null +++ b/crusoe-bastion-host/terraform/main.tf @@ -0,0 +1,86 @@ +data "crusoe_vpc_networks" "all" {} + +locals { + bastion_count = var.ha_enabled ? var.ha_count : 1 + vpc_network = [for n in data.crusoe_vpc_networks.all.vpc_networks : n if n.name == var.vpc_network][0] + vpc_network_id = local.vpc_network.id + vpc_cidr = local.vpc_network.cidr +} + +resource "crusoe_compute_instance" "bastion" { + count = local.bastion_count + + name = var.ha_enabled ? "${var.bastion_name}-${count.index + 1}" : var.bastion_name + type = var.instance_type + location = var.location + image = "ubuntu22.04:latest" + ssh_key = var.ssh_public_key + + startup_script = file("${path.module}/user-data.sh") + + disks = [ + { + id = crusoe_storage_disk.bastion_disk[count.index].id + mode = "read-write" + attachment_type = "data" + } + ] + + network_interfaces = [ + { + public_ipv4 = { + type = "dynamic" + } + } + ] +} + +resource "crusoe_storage_disk" "bastion_disk" { + count = local.bastion_count + + name = var.ha_enabled ? "${var.bastion_name}-disk-${count.index + 1}" : "${var.bastion_name}-disk" + location = var.location + size = "${var.disk_size_gib}GiB" + type = "persistent-ssd" +} + + +resource "crusoe_vpc_firewall_rule" "bastion_ssh" { + name = "${var.bastion_name}-ssh-inbound" + network = local.vpc_network_id + action = "allow" + direction = "ingress" + protocols = "tcp" + source = join(",", var.allowed_ssh_cidrs) + source_ports = "1-65535" + destination = local.vpc_cidr + destination_ports = tostring(var.ssh_port) +} + +resource "crusoe_vpc_firewall_rule" "bastion_egress_tcp_udp" { + name = "${var.bastion_name}-egress-tcp-udp" + network = local.vpc_network_id + action = "allow" + direction = "egress" + protocols = "tcp,udp" + source = local.vpc_cidr + source_ports = "1-65535" + destination = "0.0.0.0/0" + destination_ports = "1-65535" +} + +resource "crusoe_vpc_firewall_rule" "bastion_egress_icmp" { + name = "${var.bastion_name}-egress-icmp" + network = local.vpc_network_id + action = "allow" + direction = "egress" + protocols = "icmp" + source = local.vpc_cidr + source_ports = "" + destination = "0.0.0.0/0" + destination_ports = "" +} + +resource "random_id" "deployment" { + byte_length = 4 +} diff --git a/crusoe-bastion-host/terraform/outputs.tf b/crusoe-bastion-host/terraform/outputs.tf new file mode 100644 index 0000000..a6c6c3e --- /dev/null +++ b/crusoe-bastion-host/terraform/outputs.tf @@ -0,0 +1,123 @@ +output "bastion_public_ips" { + description = "Public IP addresses of bastion host(s)" + value = [for instance in crusoe_compute_instance.bastion : instance.network_interfaces[0].public_ipv4.address] +} + +output "bastion_private_ips" { + description = "Private IP addresses of bastion host(s)" + value = [for instance in crusoe_compute_instance.bastion : instance.network_interfaces[0].private_ipv4.address] +} + +output "bastion_names" { + description = "Names of bastion host(s)" + value = [for instance in crusoe_compute_instance.bastion : instance.name] +} + +output "ssh_commands" { + description = "SSH commands to connect to bastion host(s)" + value = [ + for idx, instance in crusoe_compute_instance.bastion : + "ssh -i ~/.ssh/your-private-key bastionadmin@${instance.network_interfaces[0].public_ipv4.address}" + ] +} + +output "connection_instructions" { + description = "Instructions for connecting through the bastion" + value = <<-EOT + + ╔════════════════════════════════════════════════════════════════════════════╗ + ║ BASTION HOST DEPLOYMENT COMPLETE ║ + ╚════════════════════════════════════════════════════════════════════════════╝ + + Bastion Host(s) Deployed: ${local.bastion_count} + + ${join("\n ", [for idx, instance in crusoe_compute_instance.bastion : + "Bastion ${idx + 1}:\n Name: ${instance.name}\n Public IP: ${instance.network_interfaces[0].public_ipv4.address}\n Private IP: ${instance.network_interfaces[0].private_ipv4.address}" + ])} + + ┌─────────────────────────────────────────────────────────────────────────┐ + │ CONNECT TO BASTION │ + └─────────────────────────────────────────────────────────────────────────┘ + + ssh -i ~/.ssh/your-private-key bastionadmin@${crusoe_compute_instance.bastion[0].network_interfaces[0].public_ipv4.address} + + ┌─────────────────────────────────────────────────────────────────────────┐ + │ CONNECT TO PRIVATE INSTANCE VIA BASTION │ + └─────────────────────────────────────────────────────────────────────────┘ + + # Method 1: SSH Jump Host + ssh -i ~/.ssh/your-private-key -J bastionadmin@${crusoe_compute_instance.bastion[0].network_interfaces[0].public_ipv4.address} user@ + + # Method 2: SSH ProxyJump (add to ~/.ssh/config) + Host bastion + HostName ${crusoe_compute_instance.bastion[0].network_interfaces[0].public_ipv4.address} + User bastionadmin + IdentityFile ~/.ssh/your-private-key + + Host private-instance + HostName + User ubuntu + ProxyJump bastion + IdentityFile ~/.ssh/your-private-key + + ┌─────────────────────────────────────────────────────────────────────────┐ + │ MANAGE USERS │ + └─────────────────────────────────────────────────────────────────────────┘ + + # Add a new user + cd ../scripts + ./add-user.sh + + # Remove a user + ./remove-user.sh + + # View audit logs + ./audit-logs.sh + + ┌─────────────────────────────────────────────────────────────────────────┐ + │ SECURITY NOTES │ + └─────────────────────────────────────────────────────────────────────────┘ + + ✓ SSH key-based authentication enabled + ✓ Root login disabled + ✓ Automatic security updates: ENABLED + ✓ Fail2ban intrusion prevention: ENABLED + ✓ Session logging: ENABLED + ✓ Session timeout: 900 seconds + + For more information, see the README.md and SECURITY.md files. + + EOT +} + +output "bastion_instance_ids" { + description = "Instance IDs of bastion host(s)" + value = [for instance in crusoe_compute_instance.bastion : instance.id] +} + +output "deployment_id" { + description = "Unique deployment identifier" + value = random_id.deployment.hex +} + +output "ssh_config_snippet" { + description = "SSH config snippet for ~/.ssh/config" + value = <<-EOT + # Crusoe Bastion Host Configuration + # Add this to your ~/.ssh/config file + + Host crusoe-bastion + HostName ${crusoe_compute_instance.bastion[0].network_interfaces[0].public_ipv4.address} + User bastionadmin + IdentityFile ~/.ssh/your-private-key + ServerAliveInterval 60 + ServerAliveCountMax 3 + + # Example private instance configuration + # Host my-private-instance + # HostName + # User ubuntu + # ProxyJump crusoe-bastion + # IdentityFile ~/.ssh/your-private-key + EOT +} diff --git a/crusoe-bastion-host/terraform/providers.tf b/crusoe-bastion-host/terraform/providers.tf new file mode 100644 index 0000000..5bbd68d --- /dev/null +++ b/crusoe-bastion-host/terraform/providers.tf @@ -0,0 +1,17 @@ +terraform { + required_version = ">= 1.0" + + required_providers { + crusoe = { + source = "crusoecloud/crusoe" + version = ">= 0.5.0" + } + random = { + source = "hashicorp/random" + version = ">= 3.0" + } + } +} + +provider "crusoe" { +} diff --git a/crusoe-bastion-host/terraform/terraform.tfvars.example b/crusoe-bastion-host/terraform/terraform.tfvars.example new file mode 100644 index 0000000..67c7a71 --- /dev/null +++ b/crusoe-bastion-host/terraform/terraform.tfvars.example @@ -0,0 +1,32 @@ +# Crusoe Bastion Host Configuration +# Copy this file to terraform.tfvars and fill in your values + +# Required Variables +project_id = "your-crusoe-project-id" +location = "us-east1-a" +ssh_public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC... your-email@example.com" + +# Optional Variables (uncomment to customize) + +# bastion_name = "bastion-host" +# instance_type = "c1a.2x" +# disk_size_gib = 32 + +# VPC network for firewall rules (default: "default") +# vpc_network = "default" + +# Restrict SSH access to specific IP ranges +# allowed_ssh_cidrs = [ +# "203.0.113.0/24", # Your office IP range +# "198.51.100.0/24" # Your VPN IP range +# ] + +# SSH port for firewall rules +# ssh_port = 22 + +# High Availability Configuration +# ha_enabled = false +# ha_count = 2 + +# Note: Security features (session logging, fail2ban, auto-updates) are +# configured in the user-data.sh script and enabled by default. diff --git a/crusoe-bastion-host/terraform/user-data.sh b/crusoe-bastion-host/terraform/user-data.sh new file mode 100644 index 0000000..aaa9ccf --- /dev/null +++ b/crusoe-bastion-host/terraform/user-data.sh @@ -0,0 +1,342 @@ +#!/bin/bash +# Don't use set -e - we want the script to continue even if some steps fail +# Each critical step will handle errors individually + +# Bastion Host Hardening Script +# This script is executed on first boot via cloud-init + +LOG_FILE="/var/log/bastion-setup.log" +exec 1> >(tee -a "$LOG_FILE") +exec 2>&1 + +echo "==========================================" +echo "Bastion Host Setup Started: $(date)" +echo "==========================================" + +# Configuration - can be customized +ADMIN_USERNAME="bastionadmin" +ENABLE_SESSION_LOGGING="true" +AUTO_UPDATE_ENABLED="true" +FAIL2BAN_ENABLED="true" +SSH_PORT="22" +SESSION_TIMEOUT="900" + +# Update system packages +echo "[1/10] Updating system packages..." +export DEBIAN_FRONTEND=noninteractive +apt-get update -y +apt-get upgrade -y + +# Install essential packages +echo "[2/10] Installing essential packages..." +apt-get install -y \ + curl \ + wget \ + vim \ + htop \ + net-tools \ + ufw \ + unattended-upgrades \ + apt-listchanges \ + software-properties-common + +# Create admin user +echo "[3/10] Creating admin user: $ADMIN_USERNAME..." +if ! id "$ADMIN_USERNAME" &>/dev/null; then + useradd -m -s /bin/bash -G sudo "$ADMIN_USERNAME" + echo "$ADMIN_USERNAME ALL=(ALL) NOPASSWD:ALL" > "/etc/sudoers.d/$ADMIN_USERNAME" + chmod 0440 "/etc/sudoers.d/$ADMIN_USERNAME" + + # Setup SSH key for admin user (copy from ubuntu user if exists) + mkdir -p "/home/$ADMIN_USERNAME/.ssh" + if [ -f /home/ubuntu/.ssh/authorized_keys ]; then + cp /home/ubuntu/.ssh/authorized_keys "/home/$ADMIN_USERNAME/.ssh/authorized_keys" + fi + chmod 700 "/home/$ADMIN_USERNAME/.ssh" + chmod 600 "/home/$ADMIN_USERNAME/.ssh/authorized_keys" 2>/dev/null || true + chown -R "$ADMIN_USERNAME:$ADMIN_USERNAME" "/home/$ADMIN_USERNAME/.ssh" +fi + +# Configure SSH hardening +echo "[4/10] Hardening SSH configuration..." +SSH_CONFIG="/etc/ssh/sshd_config" +cp "$SSH_CONFIG" "$SSH_CONFIG.backup" + +# SSH hardening settings +cat > /etc/ssh/sshd_config.d/99-bastion-hardening.conf < /etc/ssh/banner <<'EOF' +╔════════════════════════════════════════════════════════════════════════════╗ +║ AUTHORIZED ACCESS ONLY ║ +╚════════════════════════════════════════════════════════════════════════════╝ + +This system is a bastion host for accessing private infrastructure. +All connections are monitored and logged. +Unauthorized access is prohibited and will be prosecuted. + +EOF + +# Configure session logging if enabled +if [ "$ENABLE_SESSION_LOGGING" = "true" ]; then + echo "[5/10] Configuring session logging..." + + # Create log directory - writable by all users (sticky bit like /tmp) + mkdir -p /var/log/bastion-sessions + chmod 1777 /var/log/bastion-sessions + + # The 'script' command is part of bsdutils which is pre-installed on Ubuntu + # No additional package installation needed + + # Create session logging script + cat > /usr/local/bin/log-session.sh <<'LOGSCRIPT' +#!/bin/bash +# Log SSH sessions - errors are silently ignored to avoid messy output +SESSION_LOG_DIR="/var/log/bastion-sessions" +SESSION_LOG="$SESSION_LOG_DIR/$(date +%Y%m%d-%H%M%S)-$USER-$$-$(who am i 2>/dev/null | awk '{print $NF}' | tr -d '()').log" + +# Only log if we can write to the directory +if [ -w "$SESSION_LOG_DIR" ]; then + # Create log entry + { + echo "Session started: $(date)" + echo "User: $USER" + echo "From: $(who am i 2>/dev/null | awk '{print $NF}' | tr -d '()')" + echo "========================================" + } > "$SESSION_LOG" 2>/dev/null + + # Start script recording + exec /usr/bin/script -q -f -a "$SESSION_LOG" +fi +LOGSCRIPT + + chmod +x /usr/local/bin/log-session.sh + + # Add to bash profile for all users + echo 'if [ -n "$SSH_CONNECTION" ] && [ -x /usr/local/bin/log-session.sh ]; then /usr/local/bin/log-session.sh; fi' >> /etc/profile.d/session-logging.sh + chmod +x /etc/profile.d/session-logging.sh +else + echo "[5/10] Session logging disabled, skipping..." +fi + +# Configure automatic security updates +if [ "$AUTO_UPDATE_ENABLED" = "true" ]; then + echo "[6/10] Configuring automatic security updates..." + + cat > /etc/apt/apt.conf.d/50unattended-upgrades < /etc/apt/apt.conf.d/20auto-upgrades < /etc/fail2ban/jail.local <> /etc/sysctl.conf < /opt/bastion-scripts/add-user.sh <<'ADDUSER' +#!/bin/bash +# Add a new user to the bastion host +if [ $# -ne 2 ]; then + echo "Usage: $0 " + exit 1 +fi + +USERNAME=$1 +SSH_KEY=$2 + +useradd -m -s /bin/bash "$USERNAME" +mkdir -p "/home/$USERNAME/.ssh" +echo "$SSH_KEY" > "/home/$USERNAME/.ssh/authorized_keys" +chmod 700 "/home/$USERNAME/.ssh" +chmod 600 "/home/$USERNAME/.ssh/authorized_keys" +chown -R "$USERNAME:$USERNAME" "/home/$USERNAME/.ssh" + +# Update SSH config to allow this user +if ! grep -q "^AllowUsers.*$USERNAME" /etc/ssh/sshd_config.d/99-bastion-hardening.conf; then + sed -i "s/^AllowUsers.*/& $USERNAME/" /etc/ssh/sshd_config.d/99-bastion-hardening.conf + systemctl reload sshd +fi + +echo "User $USERNAME added successfully" +ADDUSER + +cat > /opt/bastion-scripts/remove-user.sh <<'REMOVEUSER' +#!/bin/bash +# Remove a user from the bastion host +if [ $# -ne 1 ]; then + echo "Usage: $0 " + exit 1 +fi + +USERNAME=$1 + +userdel -r "$USERNAME" 2>/dev/null || true + +# Update SSH config to remove this user +sed -i "s/ $USERNAME//" /etc/ssh/sshd_config.d/99-bastion-hardening.conf +systemctl reload sshd + +echo "User $USERNAME removed successfully" +REMOVEUSER + +chmod +x /opt/bastion-scripts/*.sh + +# Test SSH config before restarting +echo "Testing SSH configuration..." +if sshd -t 2>/dev/null; then + echo "SSH config valid, restarting service..." + systemctl restart sshd +else + echo "WARNING: SSH config test failed, not restarting sshd" + echo "Removing custom config to preserve access..." + rm -f /etc/ssh/sshd_config.d/99-bastion-hardening.conf +fi + +# Create MOTD +cat > /etc/motd <<'MOTD' +╔════════════════════════════════════════════════════════════════════════════╗ +║ CRUSOE BASTION HOST ║ +╚════════════════════════════════════════════════════════════════════════════╝ + +This is a hardened bastion host for secure access to private infrastructure. + +Management Scripts: + - Add user: sudo /opt/bastion-scripts/add-user.sh + - Remove user: sudo /opt/bastion-scripts/remove-user.sh + +Session Logs: /var/log/bastion-sessions/ +System Logs: /var/log/bastion-setup.log + +For support, see: https://github.com/crusoecloud/solutions-library + +MOTD + +# Disable the default ubuntu user for security +# Attackers commonly try default usernames like 'ubuntu' +echo "Disabling default ubuntu user for security..." +if id ubuntu &>/dev/null; then + # Lock the ubuntu account (prevents login but keeps the account for reference) + passwd -l ubuntu + # Remove ubuntu from AllowUsers is already done above + echo "Default 'ubuntu' user has been locked" + echo "Use '$ADMIN_USERNAME' for all administrative access" +fi + +echo "==========================================" +echo "Bastion Host Setup Completed: $(date)" +echo "==========================================" +echo "" +echo "IMPORTANT: Connect using: ssh $ADMIN_USERNAME@" +echo "The default 'ubuntu' user has been disabled for security." diff --git a/crusoe-bastion-host/terraform/variables.tf b/crusoe-bastion-host/terraform/variables.tf new file mode 100644 index 0000000..017053c --- /dev/null +++ b/crusoe-bastion-host/terraform/variables.tf @@ -0,0 +1,62 @@ +variable "project_id" { + description = "Crusoe Cloud project ID" + type = string +} + +variable "vpc_network" { + description = "VPC network name for firewall rules" + type = string + default = "default-vpc-network" +} + +variable "location" { + description = "Crusoe Cloud location (e.g., us-east1-a)" + type = string +} + +variable "bastion_name" { + description = "Name for the bastion host instance" + type = string + default = "bastion-host" +} + +variable "instance_type" { + description = "Instance type for the bastion host" + type = string + default = "c1a.2x" +} + +variable "disk_size_gib" { + description = "Root disk size in GiB" + type = number + default = 32 +} + +variable "ssh_public_key" { + description = "SSH public key for bastion access" + type = string +} + +variable "allowed_ssh_cidrs" { + description = "List of CIDR blocks allowed to SSH to the bastion" + type = list(string) + default = ["0.0.0.0/0"] +} + +variable "ha_enabled" { + description = "Enable high availability mode (deploy multiple bastions)" + type = bool + default = false +} + +variable "ha_count" { + description = "Number of bastion hosts to deploy in HA mode" + type = number + default = 2 +} + +variable "ssh_port" { + description = "SSH port for firewall rules (default 22)" + type = number + default = 22 +}