feat: migrate config system from dotenv to koanf/YAML #102
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -89,45 +89,46 @@ root hard nofile 65536 | |
|
|
||
| ## Configuration | ||
|
|
||
| Hypeman reads configuration from a YAML config file. See `config.example.yaml` (Linux) and `config.example.darwin.yaml` (macOS) for all available settings with comments. | ||
|
|
||
| The config file is searched in these locations (first match wins): | ||
| - Path specified by `CONFIG_PATH` environment variable | ||
| - `/etc/hypeman/config.yaml` (Linux) | ||
| - `~/.config/hypeman/config.yaml` (all platforms) | ||
|
|
||
| Common settings: | ||
|
|
||
| | Key | Description | Default | | ||
| |-----|-------------|---------| | ||
| | `port` | HTTP server port | `8080` | | ||
| | `data_dir` | Data directory for VM images, volumes, etc. | `/var/lib/hypeman` | | ||
| | `jwt_secret` | Secret key for JWT authentication (required) | _(empty)_ | | ||
| | `env` | Deployment environment (filters telemetry) | `unset` | | ||
| | `network.bridge_name` | Network bridge for VM networking | `vmbr0` | | ||
| | `network.subnet_cidr` | CIDR for the VM network subnet | `10.100.0.0/16` | | ||
| | `network.uplink_interface` | Host interface for VM internet access | _(auto-detect)_ | | ||
| | `network.dns_server` | DNS server for VMs | `1.1.1.1` | | ||
| | `caddy.listen_address` | Address for Caddy ingress listeners | `0.0.0.0` | | ||
| | `caddy.admin_address` | Address for Caddy admin API | `127.0.0.1` | | ||
| | `caddy.admin_port` | Port for Caddy admin API | `2019` | | ||
| | `caddy.stop_on_shutdown` | Stop Caddy when hypeman shuts down | `false` | | ||
| | `logging.level` | Log level (debug, info, warn, error) | `info` | | ||
| | `otel.enabled` | Enable OpenTelemetry traces/metrics | `false` | | ||
| | `otel.endpoint` | OTLP gRPC endpoint | `127.0.0.1:4317` | | ||
| | `limits.max_concurrent_builds` | Max concurrent image builds | `1` | | ||
| | `limits.max_overlay_size` | Max overlay filesystem size | `100GB` | | ||
| | `acme.email` | Email for ACME certificate registration | _(empty)_ | | ||
| | `acme.dns_provider` | DNS provider for ACME challenges | _(empty)_ | | ||
| | `acme.cloudflare_api_token` | Cloudflare API token | _(empty)_ | | ||
| | `build.docker_socket` | Path to Docker socket | `/var/run/docker.sock` | | ||
|
|
||
| Environment variables can also override any config key using `__` as the nesting separator (e.g. `CADDY__LISTEN_ADDRESS` overrides `caddy.listen_address`). | ||
|
|
||
| **Important: Subnet Configuration** | ||
|
|
||
| The default subnet `10.100.0.0/16` is chosen to avoid common conflicts. Hypeman will detect conflicts with existing routes on startup and fail with guidance. | ||
|
|
||
| If you need a different subnet, set `network.subnet_cidr` in your config file. The gateway is automatically derived as the first IP in the subnet (e.g., `10.100.0.0/16` → `10.100.0.1`). | ||
|
|
||
| **Alternative subnets if needed:** | ||
|
|
||
|
|
@@ -136,14 +137,15 @@ If you need a different subnet, set `SUBNET_CIDR` in your environment. The gatew | |
|
|
||
| **Example:** | ||
|
|
||
| ```yaml | ||
| # In your config.yaml | ||
| network: | ||
| subnet_cidr: 172.30.0.0/16 | ||
| ``` | ||
|
|
||
| **Finding the uplink interface (`network.uplink_interface`)** | ||
|
|
||
| `network.uplink_interface` tells Hypeman which host interface to use for routing VM traffic to the outside world (for iptables MASQUERADE rules). On many hosts this is `eth0`, but laptops and more complex setups often use Wi-Fi or other names. | ||
|
|
||
| **Quick way to discover it:** | ||
|
|
||
|
|
@@ -158,10 +160,11 @@ Look for the `dev` field in the output, for example: | |
| 1.1.1.1 via 192.168.12.1 dev wlp2s0 src 192.168.12.98 | ||
| ``` | ||
|
|
||
| In this case, `wlp2s0` is the uplink interface, so you would set in your config file: | ||
|
|
||
| ```yaml | ||
| network: | ||
| uplink_interface: wlp2s0 | ||
| ``` | ||
|
|
||
| You can also inspect all routes: | ||
|
|
@@ -178,15 +181,13 @@ Hypeman uses Caddy with automatic ACME certificates for TLS termination. Certifi | |
|
|
||
| To enable TLS ingresses: | ||
|
|
||
| 1. Configure ACME credentials in your `config.yaml`: | ||
|
|
||
| ```yaml | ||
| acme: | ||
| email: [email protected] | ||
| dns_provider: cloudflare | ||
| cloudflare_api_token: your-api-token | ||
| ``` | ||
|
|
||
| 2. Create an ingress with TLS enabled: | ||
|
|
@@ -205,13 +206,13 @@ curl -X POST http://localhost:8080/v1/ingresses \ | |
| }' | ||
| ``` | ||
|
|
||
| Certificates are stored in `<data_dir>/caddy/data/` and auto-renewed by Caddy. | ||
|
|
||
| ### Setup | ||
|
|
||
| ```bash | ||
| cp config.example.yaml ~/.config/hypeman/config.yaml | ||
| # Edit config.yaml and set jwt_secret and other configuration values | ||
| ``` | ||
|
|
||
| ### Data directory | ||
|
|
@@ -253,15 +254,16 @@ make gen-jwt | |
| make dev | ||
| ``` | ||
|
|
||
| The server will start on port 8080 (configurable via `port` in config.yaml). | ||
|
|
||
| ### Setting Up the Builder Image (for Dockerfile builds) | ||
|
|
||
| The builder image is required for `hypeman build` to work. When `build.builder_image` is unset or empty, the server will automatically build and push the builder image on startup using Docker. This is the easiest way to get started — just ensure Docker is available and run `make dev`. If a build is requested while the builder image is still being prepared, the server returns a clear error asking you to retry shortly. | ||
|
|
||
| On macOS with Colima, set the Docker socket path in your config file: | ||
| ```yaml | ||
| build: | ||
| docker_socket: ~/.colima/default/docker.sock | ||
| ``` | ||
|
|
||
| ### Local OpenTelemetry (optional) | ||
|
|
@@ -287,9 +289,11 @@ docker run -d --name lgtm \ | |
| # If developing on a remote server, forward the port to your local machine (or YOLO): | ||
| # ssh -L 3001:localhost:3000 your-server (then open http://localhost:3001) | ||
|
|
||
| # Enable OTel in config.yaml (set env to your name to filter your telemetry) | ||
| # Add to your config.yaml: | ||
| # otel: | ||
| # enabled: true | ||
| # env: yourname | ||
|
|
||
| # Restart dev server | ||
| make dev | ||
|
|
@@ -359,8 +363,9 @@ export PATH="/opt/homebrew/opt/e2fsprogs/bin:/opt/homebrew/opt/e2fsprogs/sbin:$P | |
| # Add to ~/.zshrc for persistence | ||
|
|
||
| # 3. Configure environment | ||
| mkdir -p ~/.config/hypeman | ||
| cp config.example.darwin.yaml ~/.config/hypeman/config.yaml | ||
| # Edit config.yaml as needed (defaults work for local development) | ||
|
|
||
| # 4. Create data directory | ||
| mkdir -p ~/Library/Application\ Support/hypeman | ||
|
|
@@ -387,16 +392,16 @@ The `make dev` command automatically detects macOS and: | |
|
|
||
| ### macOS-Specific Configuration | ||
|
|
||
| The following config keys work differently on macOS (see `config.example.darwin.yaml`): | ||
|
|
||
| | Config key | Linux | macOS | | ||
| |----------|-------|-------| | ||
| | `hypervisor.default` | `cloud-hypervisor` | `vz` | | ||
| | `data_dir` | `/var/lib/hypeman` | `~/Library/Application Support/hypeman` | | ||
| | `caddy.internal_dns_port` | `5353` | `5354` (5353 is used by mDNSResponder) | | ||
| | `network.bridge_name` | Used | Ignored (NAT) | | ||
| | `network.subnet_cidr` | Used | Ignored (NAT) | | ||
| | `network.uplink_interface` | Used | Ignored (NAT) | | ||
| | Network rate limiting | Supported | Not supported | | ||
| | GPU passthrough | Supported (VFIO) | Not supported | | ||
|
|
||
|
|
@@ -463,8 +468,8 @@ brew install caddy | |
|
|
||
| **"address already in use" on port 5353** | ||
| - Port 5353 is used by mDNSResponder (Bonjour) on macOS | ||
| - Use port 5354 instead: set `caddy.internal_dns_port: 5354` in `config.yaml` | ||
| - The `config.example.darwin.yaml` already has this configured correctly | ||
|
|
||
| **"Virtualization.framework is not available"** | ||
| - Ensure you're on macOS 11.0+ | ||
|
|
@@ -475,7 +480,7 @@ brew install caddy | |
| - Check: `sw_vers` and `uname -m` (should be arm64) | ||
|
|
||
| **VM fails to start** | ||
| - Check serial log: `<data_dir>/instances/<id>/serial.log` | ||
| - Ensure kernel and initrd paths are correct in config | ||
|
|
||
| **IOMMU/VFIO warnings at startup** | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # ============================================================================= | ||
| # Hypeman CLI Configuration | ||
| # ============================================================================= | ||
| # Place this file at ~/.config/hypeman/cli.yaml | ||
| # | ||
| # The install script automatically generates this file with a valid token | ||
| # for local access. For remote CLI access, create this file manually. | ||
| # | ||
| # Configuration precedence (highest to lowest): | ||
| # 1. CLI flags (e.g., --base-url) | ||
| # 2. Environment variables (HYPEMAN_BASE_URL, HYPEMAN_API_KEY) | ||
| # 3. This YAML config file | ||
| # ============================================================================= | ||
|
|
||
| # Hypeman API URL | ||
| base_url: http://localhost:8080 | ||
|
|
||
| # API authentication token (generated by hypeman-token) | ||
| api_key: "" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CI hardcodes temporary CLI branch name
High Severity
The e2e install test hardcodes
CLI_BRANCH=koanf-yaml-config, referencing a development branch in the companion CLI repository. Once this PR and the companion CLI PR are merged and thekoanf-yaml-configbranch is deleted, thegit clone --branchininstall.shwill fail, breaking the CI e2e install test on every subsequent run.