fix(driver): pin launchd drivers to committed code, and make an unpinned fire LOUD (#558)

.github/workflows/ci.yml

@@ -444,6 +444,27 @@ jobs:
         make install-lint
         make lint
 
+  launchd-drivers:
+    name: launchd drivers (bash 3.2)
+    # macOS DELIBERATELY, and it is the whole point of this job: the rig runs bash 3.2.57 and
+    # GitHub's macOS runners still ship 3.2 as /bin/bash, while ubuntu ships bash 5. A suite
+    # green on bash 5 proves nothing about the constraint that actually bites here — every
+    # `declare -A` or ${v,,} that reaches tools/launchd/ breaks the rig and passes on ubuntu.
+    # No Go, no cache: these are shell + git tests and nothing else.
+    runs-on: macos-latest
+
+    steps:
+    - uses: actions/checkout@v7
+
+    - name: Confirm /bin/bash really is 3.2 (guard against a silent runner upgrade)
+      run: |
+        /bin/bash --version | head -1
+        /bin/bash -c '[ "${BASH_VERSINFO[0]}" -eq 3 ]' \
+          || { echo "::error::/bin/bash is no longer 3.x — this job no longer tests the rig's shell"; exit 1; }
+
+    - name: Run launchd driver tests
+      run: make test-launchd-drivers
+
   govulncheck:
     name: govulncheck (vuln gate)
     runs-on: ubuntu-latest

changelogs/v0.18-current.md

@@ -545,6 +545,34 @@
 
 ### Mission infrastructure (no user-facing language or CLI change)
 
+- **Launchd drivers now run committed code, and failing to costs you a notification instead
+  of nothing** ([#558](https://github.com/sunholo-data/ailang/issues/558)). launchd invokes
+  each driver by absolute path into the shared clone, and *everything* a fire reads hangs off
+  that one root (`mission-control.sh:40`): the driver, the skill under `.claude/skills/`, and
+  the charter under `design_docs/`. Nothing keeps that clone current and nothing reported the
+  drift, so three artefact classes went stale together — measured as `#556`'s retired qwen3.5
+  running 24/24 two days after retirement (iter-131), a stale skill (iter-128), a stale charter
+  (iter-129), and again on 2026-08-12 when `564cc4640`'s lane-degradation fix sat inert on V1,
+  12 commits behind, while both sibling missions had it.
+
+  Previous repairs patched individual artefacts, which fixes one third of the bug. New
+  `tools/launchd/lib/pin-root.sh` pins the **root**: it re-execs the driver out of a worktree
+  pinned to committed `origin/dev`, moving script, skill and charter together. It re-points
+  `MISSION_WORKDIR` as well — `mission-motoko.env:8` and `mission-world.env:5` pin it, so
+  pinning only the script would leave those two running a fresh driver against a stale charter,
+  a half-fix that reports green. Sprint worktrees are unaffected (created by absolute path).
+
+  A failed pin is **loud, not fatal**: aborting would make network availability a hard
+  dependency of every fire, so the fire proceeds and posts *"driver ran UNPINNED"* to both human
+  channels. Silently continuing would have rebuilt the very defect `564cc4640` removed one layer
+  up. Escape hatch `AILANG_DRIVER_PIN=0`; `MISSION_DRY_RUN=1` reports `pin=<status>(<n> behind)`.
+  The two emit blocks share one `_mc_notify`, because two near-identical notifiers is how one of
+  them silently rots. First-ever automated coverage for `tools/launchd` (37 tests, `make
+  test-launchd-drivers`, bash 3.2.57): both pin arms, both notify arms, the fetch-failure and
+  driver-missing-from-ref refusals, and a control proving the stale clone really was stale.
+  Stated plainly: this cannot help until the shared clone receives it once, and that first
+  reconcile is human (Principle 0).
+
 - **`dev.ailang.mission-recovery` — a probe-stall costs ~20 min of loop time instead of
   90.** When every controller probe times out the driver refuses, and with
   `StartInterval` at 5400s that slot is gone for a full 90 minutes even if the stall

design_docs/v1-mission.md

@@ -1654,6 +1654,43 @@ frozen; contracts projection live).
   evaluator when the gemini lane is unavailable). Rationale: the opus bucket dried Thursday
   this week at ~55% duty cycle; these two move the remaining Anthropic-heavy sub-agent roles
   off-bucket so the week stretches toward Monday-to-Monday.
+- **[BLOCKED-ON-EVIDENCE — do NOT pick before the gate below is measured] m-driver-pin-rollout**
+  (`#558`; mission infra, not a language change; does not gate v1.0). **Prerequisite already
+  shipped**: PR `#666` (2026-08-12, attended) adds `tools/launchd/lib/pin-root.sh` and wires it
+  into `mission-control.sh` ONLY — the driver re-execs from a worktree pinned to committed
+  `origin/dev`, moving the driver, the skill and the charter together, and posting *"driver ran
+  UNPINNED"* on both human channels when the pin fails. This item is the rollout to the
+  remaining **five** entry points: `nightly-eval.sh`, `nightly-lang-eval.sh`,
+  `mission-recovery.sh`, `os-rotation-filler.sh`, `rig-watchdog.sh`.
+
+  **Why one root, not five patches** (Principle 3): everything a fire reads hangs off `REPO` at
+  `mission-control.sh:40`, so the driver (`$0`), the skill (`cwd`) and the charter (`cwd`) go
+  stale *together*. Measured four times — `#556`'s retired qwen3.5 running 24/24 (iter-131), a
+  stale skill (iter-128), a stale charter (iter-129), and `564cc4640` inert on V1 at 12 commits
+  behind (2026-08-12) while both sibling missions had it. Two one-time human reconciles, zero
+  durable fixes, until `#666`.
+
+  **INERT UNTIL RECONCILED, and say so rather than reporting the capability as gained** — the
+  same trap that deferred `m-planner-codex-lane` at iter-131. `#666` cannot take effect until
+  the shared clone receives it once, and that is a human branch op (the standing fast-forward
+  authorisation above does NOT cover it: local dev is 1 ahead, not 0). **Order matters:** merge
+  `#666` FIRST, reconcile SECOND — reconciling first brings the lane fix but not the pin, and
+  the clone simply starts drifting again.
+
+  **THE GATE (evidence, not a delay):** ≥3 consecutive V1 fires logging `driver pin: running
+  committed origin/dev @ <sha>` with a normal iteration completing. Read it from
+  `/tmp/ailang-mission-control.log`, not from the file's presence on disk — measuring the wrong
+  copy is the same class of error as the bug. **Known limit of that gate, stated up front:** three
+  green fires exercise the re-exec, the root move and the fetch, but NOT the failure path, which
+  production will not produce on demand. That arm is covered only by `make test-launchd-drivers`
+  (37 assertions, bash 3.2 CI job) — so a passing gate is evidence about the happy path alone and
+  must not be reported as whole-fix confidence.
+
+  **Not in scope, and not ours to schedule:** `ailang-world` is a **different repo**
+  (`sunholo-data/ailang-world`) whose driver is a hand-synced *fork* — 513 lines against this
+  repo's 671, with a differently-shaped fallback site. It can never take this by copy, only by
+  port. Handed over on the cross-mission channel as `msg_20260812_085746`; World's loop decides
+  when. Do not "fix" it from here.
 - **[QUORUM-BLOCKED 2026-08-11 (iter-179) — doc LANDED, PR `#657` → squash `0a84f5377`, Gate 3b GREEN
   (4/4 required, `checks=20`, zero not-green). **Reclassified P0**: `#616` is a STATIC EFFECT-SOUNDNESS
   hole, not the DX/error-message item it was queued as — a function with NO effect annotation calling a

make/test.mk

@@ -7,7 +7,7 @@
 .PHONY: test-operator-assertions test-regression-guards test-builtin-consistency
 .PHONY: test-stdlib-canaries test-row-properties test-golden-types test-repl-smoke
 .PHONY: test-sim-stub test-stdlib-freeze verify-no-shim verify-lowering
-.PHONY: test-nightly-classifier
+.PHONY: test-nightly-classifier test-launchd-drivers
 
 # Core tests. Depends on build so integration tests that shell out to the
 # ailang binary never see a stale bin/ailang — a stale binary caused phantom
@@ -32,6 +32,15 @@ test: build ## Run all Go unit tests (builds bin/ailang first)
 test-nightly-classifier: ## Run nightly variance-guard contract and replay tests
 	@python3 tools/test_nightly_classify.py -v
 
+# The launchd drivers carried ZERO automated coverage until #558's second recurrence — a large
+# part of why two silent-staleness bugs shipped unnoticed. /bin/bash explicitly, not $$SHELL:
+# the rig runs 3.2.57, so a suite that only passes under a newer bash proves nothing about it.
+test-launchd-drivers: ## Run launchd driver tests (pin-root + degradation notices, bash 3.2)
+	@/bin/bash tools/launchd/test_pin_root.sh
+	@/bin/bash tools/launchd/test_driver_notify.sh
+	@for f in tools/launchd/*.sh tools/launchd/lib/*.sh; do /bin/bash -n "$$f" || exit 1; done
+	@echo "launchd drivers: tests + bash 3.2 syntax OK"
+
 test-parser: ## Run parser tests only
 	@echo "Testing parser..."
 	@$(GOTEST) ./internal/parser

tools/launchd/lib/pin-root.sh

@@ -0,0 +1,166 @@
+#!/usr/bin/env bash
+# pin-root.sh — make a launchd driver run COMMITTED code, and make failing to do so LOUD.
+#
+# THE CLASS THIS CLOSES (#558; measured twice — 2026-08-03 and 2026-08-12).
+# launchd invokes each driver by absolute path into a shared, mutable clone that nothing
+# keeps current, and everything the fire reads hangs off that one root:
+#
+#     REPO="${MISSION_WORKDIR:-$(cd "$(dirname "$0")/../.." && pwd)}"; cd "$REPO"
+#
+#   * the DRIVER itself         -> #556 retired qwen3.5 and the nightly ran it 24/24 two days
+#                                  later; 564cc4640's lane fix was inert on V1 while both
+#                                  sibling missions had it
+#   * the SKILL (.claude/skills) -> stale skill, iteration 128
+#   * the CHARTER (design_docs)  -> stale charter, iteration 129
+#
+# Three symptoms, one root. Patching any single artefact fixes one third of a bug, so this
+# pins the ROOT: re-exec the driver out of a worktree pinned to a committed ref, which moves
+# the script, the skill and the charter together in one move.
+#
+# WHY THE FAILURE PATH IS LOUD RATHER THAN SILENT. If the fetch fails and we quietly carry on
+# from the working tree, we have rebuilt the exact defect 564cc4640 removed one layer up — a
+# fallback whose only witness is a log nobody reads (Critical Principle 2). So a failed pin is
+# NOT fatal (making network availability a hard dependency of every fire trades a rare silent
+# staleness for a common loud outage) but it MUST be reported on the human channel by the
+# caller. This file sets PIN_STATUS=STALE and returns 1; emitting is the caller's job, because
+# only the caller knows its own early-exit points — a fire that never runs must never post.
+#
+# CONTRACT
+#   source, never execute.  $0 stays the CALLER's path, which is what makes $0-relative
+#   resolution below refer to the driver rather than to this file.
+#
+#   in    $0                          the driver's path (implicit)
+#         AILANG_DRIVER_REF           ref to pin to           (default origin/dev)
+#         AILANG_DRIVER_PIN=0         opt out entirely        (default on)
+#         AILANG_DRIVER_FETCH_TIMEOUT bounded fetch, seconds  (default 120)
+#         AILANG_DRIVER_PIN_DIR       override worktree path
+#         MISSION_NAME                namespaces the worktree so missions never collide
+#   out   PIN_STATUS  pinned | disabled | STALE
+#         PIN_NOTE    one-line human summary, safe to log or post
+#         PIN_DRIFT   how many commits the source clone is behind the ref ("?" if unknown)
+#   exit  re-execs on success and NEVER RETURNS; returns 0 (already pinned / opted out) or
+#         1 (STALE — caller continues on the working tree, loudly)
+#
+# CHICKEN AND EGG, stated rather than hidden: this block lives in the driver it protects, so it
+# does nothing until the shared clone receives it once. That first reconcile is human (Principle
+# 0 forbids unattended branch ops on a shared dirty tree). Afterwards it is self-maintaining.
+#
+# Portable to macOS bash 3.2.57 — no associative arrays, no ${v,,}, no GNU timeout.
+
+PIN_STATUS="unknown"
+PIN_NOTE=""
+PIN_DRIFT="?"
+
+# _pin_bounded SECONDS CMD... — hard wall-clock cap; rc = CMD's rc, or 124 on expiry.
+# Deliberately duplicates mission-control.sh's _mc_bounded rather than depending on it: this
+# file has to be sourceable by drivers that define no such helper, and the rig has no GNU
+# timeout (mission-control.sh:37). An unbounded fetch here would hang the fire before the
+# driver's own watchdog exists — the failure mode _mc_bounded was written for.
+_pin_bounded() {
+  local secs="$1"; shift
+  local out_f rc deadline pid
+  out_f=$(mktemp -t pin_bounded) || { PIN_BOUNDED_OUT=""; return 125; }
+  ( exec "$@" ) >"$out_f" 2>&1 &
+  pid=$!
+  deadline=$(( $(date +%s) + secs ))
+  while kill -0 "$pid" 2>/dev/null; do
+    if [ "$(date +%s)" -ge "$deadline" ]; then
+      kill "$pid" 2>/dev/null; sleep 1; kill -9 "$pid" 2>/dev/null
+      PIN_BOUNDED_OUT="$(cat "$out_f" 2>/dev/null)"; rm -f "$out_f"
+      return 124
+    fi
+    sleep 1
+  done
+  wait "$pid"; rc=$?
+  PIN_BOUNDED_OUT="$(cat "$out_f" 2>/dev/null)"; rm -f "$out_f"
+  return "$rc"
+}
+
+_pin_stale() {  # $1 = reason -> STALE, caller reports and continues on the working tree
+  PIN_STATUS="STALE"
+  PIN_NOTE="$1"
+  return 1
+}
+
+# pin_root_to_committed_ref "$@" — pass the driver's own args so the re-exec forwards them.
+pin_root_to_committed_ref() {
+  if [ -n "${AILANG_DRIVER_PINNED:-}" ]; then
+    # we ARE the re-exec'd copy; the values below crossed the exec in the environment
+    PIN_STATUS="pinned"
+    PIN_DRIFT="${AILANG_DRIVER_DRIFT:-?}"
+    PIN_NOTE="running committed ${AILANG_DRIVER_REF:-origin/dev} @ ${AILANG_DRIVER_PINNED} from ${MISSION_WORKDIR:-?} (source clone ${AILANG_DRIVER_SRC:-?} was ${PIN_DRIFT} behind)"
+    return 0
+  fi
+
+  if [ "${AILANG_DRIVER_PIN:-1}" = "0" ]; then
+    PIN_STATUS="disabled"
+    PIN_NOTE="pin disabled via AILANG_DRIVER_PIN=0 — running the working tree as-is"
+    return 0
+  fi
+
+  local ref src script wt target short drift fetch_s rc
+  ref="${AILANG_DRIVER_REF:-origin/dev}"
+  fetch_s="${AILANG_DRIVER_FETCH_TIMEOUT:-120}"
+  script=$(basename "$0")
+
+  src=$(cd "$(dirname "$0")/../.." 2>/dev/null && pwd)
+  if [ -z "$src" ]; then
+    _pin_stale "cannot resolve the source clone from \$0=$0"; return 1
+  fi
+  if ! git -C "$src" rev-parse --git-dir >/dev/null 2>&1; then
+    _pin_stale "source clone $src is not a git repository"; return 1
+  fi
+
+  _pin_bounded "$fetch_s" git -C "$src" fetch --quiet origin; rc=$?
+  if [ "$rc" -eq 124 ]; then
+    _pin_stale "git fetch origin exceeded ${fetch_s}s in $src"; return 1
+  elif [ "$rc" -ne 0 ]; then
+    _pin_stale "git fetch origin failed (rc=$rc) in $src: $(printf '%s' "${PIN_BOUNDED_OUT:-}" | tail -c 200 | tr '\n' ' ')"; return 1
+  fi
+
+  target=$(git -C "$src" rev-parse "$ref" 2>/dev/null)
+  if [ -z "$target" ]; then
+    _pin_stale "cannot resolve $ref in $src"; return 1
+  fi
+  short=$(git -C "$src" rev-parse --short "$target" 2>/dev/null)
+  drift=$(git -C "$src" rev-list --count "HEAD..$ref" 2>/dev/null)
+  [ -n "$drift" ] || drift="?"
+  PIN_DRIFT="$drift"
+
+  wt="${AILANG_DRIVER_PIN_DIR:-$HOME/.ailang-driver-pin/${MISSION_NAME:-$(basename "$script" .sh)}}"
+
+  # Refresh (or create) the pin worktree. It is a throwaway checkout with no user work in it,
+  # kept outside every dev clone, so --force can never overwrite in-progress changes.
+  git -C "$src" worktree prune >/dev/null 2>&1
+  if git -C "$src" worktree list --porcelain 2>/dev/null | grep -qx "worktree $wt"; then
+    if ! git -C "$wt" checkout --quiet --detach --force "$target" 2>/dev/null; then
+      _pin_stale "pin worktree checkout $short failed at $wt"; return 1
+    fi
+  else
+    mkdir -p "$(dirname "$wt")"
+    if ! git -C "$src" worktree add --quiet --detach --force "$wt" "$target" 2>/dev/null; then
+      _pin_stale "pin worktree add $short failed at $wt"; return 1
+    fi
+  fi
+
+  # Refuse to exec into a driver the ref does not have — a rename on dev would otherwise turn
+  # a pin into a silent no-run. Control for the whole worktree step in one assertion.
+  if [ ! -f "$wt/tools/launchd/$script" ]; then
+    _pin_stale "$ref has no tools/launchd/$script — refusing to re-exec into a missing driver"; return 1
+  fi
+
+  # MISSION_WORKDIR moves too, and that is the point: mission-control.sh:40 reads it AHEAD of
+  # $0-relative resolution, so pinning only the script would leave motoko and World (whose env
+  # files pin MISSION_WORKDIR — mission-motoko.env:8, mission-world.env:5) running a pinned
+  # driver against a stale charter and skill. That half-fix reports green, which is worse than
+  # no fix. Sprint worktrees are unaffected: the skill creates them by absolute path
+  # (mission-control SKILL.md:1662), not relative to cwd.
+  AILANG_DRIVER_PINNED="$short"
+  AILANG_DRIVER_SRC="$src"
+  AILANG_DRIVER_DRIFT="$drift"
+  AILANG_DRIVER_REF="$ref"
+  MISSION_WORKDIR="$wt"
+  export AILANG_DRIVER_PINNED AILANG_DRIVER_SRC AILANG_DRIVER_DRIFT AILANG_DRIVER_REF MISSION_WORKDIR
+
+  exec /bin/bash "$wt/tools/launchd/$script" "$@"
+}