Improvements for infra setup

[LKajan]

This started with the creation of a new test instance. The AMI previously used for the bastion server was no longer available, so it had to be updated. The update causes the bastion to be replaced, which would result in SSH host key verification errors for users.

This PR addresses the issue by using a pre-generated host key.

It also includes some other minor enhancements.

Copilot generated summary:

This pull request introduces improvements to the bastion host provisioning process, enhances Lambda logging, updates documentation for clarity and consistency, and makes a few other infrastructure tweaks. The most significant changes are the switch to using cloud-init and SSM for bastion host key management, improved security and maintainability for the bastion instance, and standardized handling of variable files across the codebase.

Bastion Host Provisioning and Security:

• Switched bastion host provisioning to use cloud-init via the cloudinit Terraform provider, replacing the previous user data template. This enables modular, maintainable configuration and secure host key management. (infra/bastion.tf, infra/bastion_config/cloud-config.yaml.tftpl, infra/bastion_config/host_key_setup.sh, infra/.terraform.lock.hcl, [1] [2] [3] [4]
• Bastion host now fetches its SSH Ed25519 host key from AWS SSM Parameter Store at boot, ensuring persistent and secure host identity across reboots. (infra/bastion_config/host_key_setup.sh, infra/bastion_config/host_key_setup.shR1-R37)
• Added a dedicated IAM policy to allow the bastion EC2 instance to read its host key from SSM, improving security and least-privilege access. (infra/iam.tf, infra/iam.tfR195-R243)

Lambda Function Improvements:

• Enabled structured JSON logging and set log levels to INFO for all Lambda functions, improving observability and log analysis. (infra/lambda.tf, [1] [2] [3] [4]

Documentation and Variable File Handling:

• Updated documentation to consistently reference variable files in the new var-files/ directory, and clarified deployment and teardown instructions. (infra/README.md, [1] [2] [3] [4]
• Updated Makefile commands to use the var-files/ directory for encrypted and decrypted tfvars files, ensuring consistency and reducing risk of misconfiguration. (infra/Makefile, infra/MakefileL73-R77)

Other Infrastructure Tweaks:

• Reduced API Gateway Lambda integration timeout to 29 seconds, likely to better align with AWS limits or application requirements. (infra/api.tf, infra/api.tfL79-R79)
• Removed the obsolete infra/bastion_user_data.tpl file, as its functionality is now covered by cloud-init. (infra/bastion_user_data.tpl, infra/bastion_user_data.tplL1-L32)

[Rikuoja]

Didn't understand what the coordinate change thing has to do with anything and what it does. Otherwise, looks very good!