-
Notifications
You must be signed in to change notification settings - Fork 223
Adds hack/dev-branch.sh #775
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| --- | ||
| name: dev-branch | ||
| description: Parallel development using hack/dev-branch.sh to isolate worktrees and Kind test clusters per feature. | ||
| --- | ||
|
|
||
| # Multi-Agent Parallel Development with `hack/dev-branch.sh` | ||
|
|
||
| This skill provides workflow guidance for executing concurrent development tasks, | ||
| working on separate features within the `agent-substrate/substrate` repository. | ||
|
|
||
| By isolating each feature inside its own **Git worktree** and **Kind Kubernetes | ||
| cluster**, multiple agents can build, test, and debug features in parallel | ||
| without step-on or port/cluster resource contention. | ||
|
|
||
| --- | ||
|
|
||
| ## Architecture & Principles | ||
|
|
||
| | Resource | Isolation Strategy | Purpose | | ||
| | :--- | :--- | :--- | | ||
| | **Source Code** | Git Worktree (`.worktrees/<branch>`) | Independent code branch & checkout per agent | | ||
| | **Kubernetes Cluster** | Kind Cluster (`kind-dev-<branch>`) | Dedicated cluster running `ate-system` per agent | | ||
| | **Container Registry** | Shared Docker Registry (`kind-registry:5001`) | Centralized image storage shared safely across clusters | | ||
| | **Environment Vars** | `.dev-env.sh` / `dev-branch.sh exec` | Auto-routes `kubectl` & E2E tests to the correct cluster | | ||
|
|
||
| --- | ||
|
|
||
| ## Step-by-Step Workflow | ||
|
|
||
| ### 1. Provision Environment | ||
|
|
||
| When starting a feature or delegating a task to a subagent, provision an | ||
| isolated environment: | ||
|
|
||
| ```bash | ||
| ./hack/dev-branch.sh setup <branch-name> | ||
| ``` | ||
|
|
||
| - **Branch Name Format**: Use distinct, descriptive names (e.g., | ||
| `agent-router-fix`, `agent-valkey-backup`). | ||
| - **Path**: Created automatically under `.worktrees/<sanitized-branch>`. | ||
|
|
||
| ### 2. Spawning Subagents (Parallel Execution) | ||
|
|
||
| When spawning parallel subagents using `invoke_subagent`: | ||
| - Pass the target worktree path (`.worktrees/<branch-name>`) in the prompt. | ||
| - Instruct subagents to use `./hack/dev-branch.sh exec <branch-name> -- | ||
| <command>` for running commands against their specific environment. | ||
|
|
||
| Example prompt fragment: | ||
|
|
||
| > You are assigned to implement feature X in branch `agent-feature-x`. | ||
| > Work inside the directory `.worktrees/agent-feature-x`. | ||
| > Run tests using `./hack/dev-branch.sh exec agent-feature-x -- ./hack/run-e2e-kind.sh`. | ||
|
|
||
| ### 3. Executing Commands & Testing | ||
|
|
||
| Instead of manually setting environment variables in every subshell, use `exec`: | ||
|
|
||
| ```bash | ||
| # Run unit tests inside the worktree | ||
| ./hack/dev-branch.sh exec <branch-name> -- go test ./... | ||
|
|
||
| # Run E2E tests against the branch's dedicated Kind cluster | ||
| ./hack/dev-branch.sh exec <branch-name> -- ./hack/run-e2e-kind.sh | ||
| ``` | ||
|
|
||
| Alternatively, `eval` the environment in the subshell: | ||
| ```bash | ||
| cd .worktrees/<branch-name> | ||
| eval "$(/path/to/main/hack/dev-branch.sh env <branch-name>)" | ||
| ./hack/run-e2e-kind.sh | ||
| ``` | ||
|
|
||
| ### 4. Avoiding Resource & Port Conflicts | ||
|
|
||
| - **Kubernetes Contexts**: Contexts are automatically isolated per cluster | ||
| (`kind-kind-dev-<branch-name>`). | ||
| - **Port Forwarding**: When port-forwarding services (e.g. `atenet-router`), | ||
| select dynamic host ports or unique ports per agent to avoid collisions: | ||
|
|
||
| ```bash | ||
| # Use random available host port | ||
| kubectl --context kind-kind-dev-<branch-name> port-forward -n ate-system svc/atenet-router :80 | ||
| ``` | ||
|
|
||
| ### 5. Listing Active Environments | ||
|
|
||
| Check all active parallel environments: | ||
|
|
||
| ```bash | ||
| ./hack/dev-branch.sh list | ||
| ``` | ||
|
|
||
| ### 6. Teardown | ||
|
|
||
| When work on a feature is finished and merged/submitted: | ||
|
|
||
| ```bash | ||
| ./hack/dev-branch.sh teardown <branch-name> | ||
| ``` | ||
|
|
||
| - Deletes the `kind-dev-<branch-name>` cluster. | ||
| - Removes the `.worktrees/<branch-name>` directory. | ||
| - Preserves the shared `kind-registry` container if other agents' Kind clusters | ||
| are still active. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -40,8 +40,13 @@ fi | |
| if [ "${reg_exists}" == true ]; then | ||
| reg_created_by="$(docker inspect --format '{{index .Config.Labels "created-by"}}' "${reg_name}" 2>/dev/null)" | ||
| if [ "${reg_created_by}" == "agent-substrate" ]; then | ||
| echo "Deleting registry container '${reg_name}' (created by us)..." | ||
| docker rm -f "${reg_name}" || true | ||
| remaining_clusters="$("${ROOT}"/hack/kind.sh get clusters 2>/dev/null | grep -v 'No kind clusters found' | grep -v '^$' || true)" | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this script was intended to be the counter to create-kind-cluster.sh, I think if we're going to delete all clusters that should be optional or a mode currently if you are doing other things on the same host with kind, we won't delete that, just the one you requested. now this PR wipes unrelated clusters
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This change in particular I think needs to be refined. Why can't an agent just tear down its own cluster? |
||
| if [ -z "${remaining_clusters}" ]; then | ||
| echo "No remaining Kind clusters found. Deleting registry container '${reg_name}'..." | ||
| docker rm -f "${reg_name}" || true | ||
| else | ||
| echo "Other Kind clusters are still running, keeping registry container '${reg_name}'." | ||
| fi | ||
| else | ||
| echo "Registry container '${reg_name}' was not created by us (${reg_created_by}), leaving it running." | ||
| fi | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.