Skip to content

fix(bin): relaunch secondmates at fleet startup only when work is pending - #13

Merged
brchue-ux merged 7 commits into
mainfrom
fm/startup-firstmate-only
Aug 9, 2026
Merged

fix(bin): relaunch secondmates at fleet startup only when work is pending#13
brchue-ux merged 7 commits into
mainfrom
fm/startup-firstmate-only

Conversation

@brchue-ux

Copy link
Copy Markdown
Owner

Intent

Fleet startup must launch the first mate and nothing else - no second mate is ever auto-started just because it is registered or happened to be open when the fleet stopped.

Captain decision 2026-08-03, on startup-token-cost grounds: 'fleet startup should never be anything more than first mate. I will manually use firstmate to call any secondmates, thus reduces startup cost.' Clarified: 'if there is ongoing work that was paused due to a crash or limit hit, those can restart. but if theres no work pending. we dont open an idle secondmate just for it to use startup tokens and perhaps not get used that session.'

THE RULE: pending work is the test. A secondmate with work paused mid-flight (crash, quota limit, killed session) IS relaunched so that work resumes. A secondmate with an empty queue is NOT opened, regardless of registration or of having been open at the stop. The captain opens the rest himself.

IMPORTANT HISTORY - this is the THIRD position on this question, and the reviewer will find superseded quotes. On 2026-07-31 the captain explicitly REJECTED a work-in-flight test and set the rule to 'reopen exactly the mates that were OPEN at the stop; busy-versus-idle is NOT the test', because he wanted idle-but-open mates back. On 2026-08-03 he reversed that on cost grounds. Quotes still sitting in data/captain.md history reflect the superseded 2026-07-31 rule and must NOT be used to argue the new behavior is wrong. Deliberately reintroducing relaunch-when-idle would be the bug.

Pending work is judged ONLY from that secondmate's own durable records, never from whether its pane happened to be alive: a non-blank line under '## Queued' or '## In flight' in that home's data/backlog.md, or any *.meta file left under that home's state/ by a child task it dispatched and never tore down. Those exact section headings are the repo-wide canonical shape (bin/fm-backlog-handoff.sh seeds every secondmate home with them; fm-session-start.sh and fm-fleet-snapshot.sh parse the same strings). This is deliberately a cheap presence test, not the structured tasks-axi backlog schema - it only decides whether a launch is worth the startup tokens.

Deliberate design decisions a reviewer should not flag as mistakes:

  • A confirmed-dead endpoint husk is still killed BEFORE the pending-work check. Cleaning up a dead husk is correct regardless of whether we relaunch.
  • When the home cannot be resolved or does not exist, the sweep emits a SECONDMATE_LIVENESS skip rather than guessing. Refusing to guess is intended: guessing wrong either burns tokens on an idle mate or strands a busy one.
  • The correctly-left-down-idle case is SILENT by design, surfacing only as a BOOTSTRAP_INFO fact under FM_BOOTSTRAP_VERBOSE_FACTS=1. It is the expected outcome, not a problem, so it must not produce an actionable diagnostic line.
  • Ambiguous/unreadable liveness reads are still preserved untouched, unchanged from before.
  • Homes, backlogs, clones, leases and registry entries all stay. This is launch policy, not retirement.

Scope covered: bin/fm-bootstrap.sh (the secondmate_home_has_pending_work helper plus the liveness sweep), AGENTS.md sections 3 and 5, the secondmate-provisioning skill's Recovery section (the single owner of the pending-work test, which every other surface cross-references rather than restating, per this repo's one-owner rule), the bootstrap-diagnostics skill's SECONDMATE_LIVENESS entry, docs/architecture.md and docs/configuration.md, and tests.

Also in this branch, deliberately: the branch was rebased from an eight-commit-stale base onto current main, resolving an AGENTS.md conflict where main had grown the bootstrap sweep list from five to six entries. Two small follow-up commits: one aligning the bootstrap-diagnostics playbook, which still told the agent to investigate any non-live registered secondmate (now wrong - an idle mate being down is correct), and one dropping an unused 'local home=$1' in the new helper.

Known and intentionally out of scope: a separate queued backlog item 'secondmate-no-autoload' (a per-mate opt-out list) is made moot by this pending-work test, but is NOT touched here and will be reconciled separately. Also noted but deliberately not addressed: a captain-held item sitting under '## Queued' still counts as pending work and will relaunch its mate.

Verification already run locally, all green: bin/fm-lint.sh exit 0 (shellcheck 0.11.0 pinned), tests/fm-secondmate-liveness.test.sh (including four new pending-work cases that drive real bootstrap runs against a fake tmux and assert no new-window for an idle mate), tests/fm-secondmate-sync.test.sh, and bin/fm-doc-audience-check.sh.

What Changed

  • bin/fm-bootstrap.sh gains secondmate_home_has_pending_work and rewires the session-start liveness sweep around it: a confirmed-dead husk is still killed, but the respawn now runs only when that secondmate's own durable records show pending work — a non-blank line under ## Queued or ## In flight in its home data/backlog.md, or a leftover *.meta under its home state/. A registered-but-idle mate is left down silently (visible only as a BOOTSTRAP_INFO fact under FM_BOOTSTRAP_VERBOSE_FACTS=1), and an unresolvable, unvalidated, or unreadable home/backlog emits a SECONDMATE_LIVENESS: skip instead of guessing.
  • Left-down mates are tracked in a new SECONDMATE_LEFT_DOWN_IDS hand-off so the sweeps that follow still fast-forward the home and propagate inherited config, but never send into the dead-by-design endpoint: no NUDGE_SECONDMATES: or CONFIG_REREAD: line, no queued reread generation, and any pre-existing nudge marker is retired rather than carried into the reopening session.
  • Documentation follows the new launch policy across AGENTS.md (sections 3 and 5), README.md, docs/architecture.md, docs/configuration.md, the secondmate-provisioning skill's Recovery section (sole owner of the pending-work test) and the bootstrap-diagnostics SECONDMATE_LIVENESS entry; tests/fm-secondmate-liveness.test.sh (21 cases) and tests/fm-secondmate-sync.test.sh (22 cases) cover the idle-left-down, in-flight-relaunch, refuse-to-guess, and left-down-not-nudged paths and pass in full.

Risk Assessment

✅ Low: Every finding from the prior rounds is now resolved with minimal, correctly-placed changes - the marker retirement sits after the id-safety and filename guards so it can only delete the canonical path, it rests on a premise the repo's own owner skill states explicitly (a fresh launch reads its instruction surface from disk), and the three-session path from the finding is covered end to end by a new test - leaving a well-bounded change with no outstanding concerns.

Testing

Ran the two suites the intent names plus the direct bootstrap and documentation suites, then drove a real end-to-end fleet startup against a fake fleet of four dead-but-registered secondmates to show the policy as a captain would experience it: the idle mate is left down with its dead husk still cleaned up and no actionable diagnostic, while the mates holding a queued item, an in-flight item, or leftover child task metadata are each relaunched, and the verbose run surfaces the left-down outcome only as a BOOTSTRAP_INFO fact. This is a shell CLI startup sweep with no rendered UI surface, so the reviewer-visible artifact is the annotated startup transcript rather than a screenshot. Everything relevant to the change is green; the single failure seen is a pre-existing orca assertion in tests/fm-bootstrap.test.sh that collides with this host's GNOME screen reader and reproduces unchanged at the base commit.

Evidence: Fleet startup transcript — idle secondmate left down, pending-work mates relaunched (both verbosity modes)
########## RUN 1: default verbosity - what the captain actually sees ##########

================================================================
 FLEET STATE AT STARTUP - four secondmates registered, all down
================================================================

--- sm-idle ---
  data/backlog.md:
    | ## In flight
    | 
    | ## Queued
    | 
    | ## Done
    | - [x] shipped-last-week - all done
  state/*.meta (child tasks it dispatched): (none)

--- sm-queued ---
  data/backlog.md:
    | ## In flight
    | 
    | ## Queued
    | - [ ] wire-up-report - captain queued this before the crash
    | 
    | ## Done
  state/*.meta (child tasks it dispatched): (none)

--- sm-inflight ---
  data/backlog.md:
    | ## In flight
    | - [ ] migrate-db - paused mid-flight when the session hit its quota limit
    | 
    | ## Queued
    | 
    | ## Done
  state/*.meta (child tasks it dispatched): (none)

--- sm-crashed ---
  data/backlog.md:
    | ## In flight
    | 
    | ## Queued
    | 
    | ## Done
  state/*.meta (child tasks it dispatched): child1.meta 

  tmux windows before startup (agent pane command per window):
    main claude
    fm-sm-idle zsh
    fm-sm-queued zsh
    fm-sm-inflight zsh
    fm-sm-crashed zsh

================================================================
 RUNNING FLEET STARTUP: bin/fm-bootstrap.sh
================================================================
  (default verbosity - every actionable secondmate line the captain sees)
    <none - the correct outcome is silent; no actionable diagnostic>

================================================================
 WHAT THE FLEET ACTUALLY LAUNCHED
================================================================
  tmux calls made by the startup sweep:
    kill-window kill-window -t =firstmate:=fm-sm-crashed
    new-window new-window -dP -F #{window_id} -t firstmate: -n fm-sm-crashed -c <world>/sm-crashed
    kill-window kill-window -t =firstmate:=fm-sm-idle
    kill-window kill-window -t =firstmate:=fm-sm-inflight
    new-window new-window -dP -F #{window_id} -t firstmate: -n fm-sm-inflight -c <world>/sm-inflight
    kill-window kill-window -t =firstmate:=fm-sm-queued
    new-window new-window -dP -F #{window_id} -t firstmate: -n fm-sm-queued -c <world>/sm-queued

  tmux windows after startup:
    main claude
    fm-sm-crashed claude
    fm-sm-inflight claude
    fm-sm-queued claude

  verdict:
    sm-idle        left down
    sm-queued      RELAUNCHED
    sm-inflight    RELAUNCHED
    sm-crashed     RELAUNCHED

########## RUN 2: same fleet under FM_BOOTSTRAP_VERBOSE_FACTS=1 ##########

================================================================
 FLEET STATE AT STARTUP - four secondmates registered, all down
================================================================

--- sm-idle ---
  data/backlog.md:
    | ## In flight
    | 
    | ## Queued
    | 
    | ## Done
    | - [x] shipped-last-week - all done
  state/*.meta (child tasks it dispatched): (none)

--- sm-queued ---
  data/backlog.md:
    | ## In flight
    | 
    | ## Queued
    | - [ ] wire-up-report - captain queued this before the crash
    | 
    | ## Done
  state/*.meta (child tasks it dispatched): (none)

--- sm-inflight ---
  data/backlog.md:
    | ## In flight
    | - [ ] migrate-db - paused mid-flight when the session hit its quota limit
    | 
    | ## Queued
    | 
    | ## Done
  state/*.meta (child tasks it dispatched): (none)

--- sm-crashed ---
  data/backlog.md:
    | ## In flight
    | 
    | ## Queued
    | 
    | ## Done
  state/*.meta (child tasks it dispatched): child1.meta 

  tmux windows before startup (agent pane command per window):
    main claude
    fm-sm-idle zsh
    fm-sm-queued zsh
    fm-sm-inflight zsh
    fm-sm-crashed zsh

================================================================
 RUNNING FLEET STARTUP: bin/fm-bootstrap.sh
================================================================
  (FM_BOOTSTRAP_VERBOSE_FACTS=1 - showing the per-secondmate facts)
  BOOTSTRAP_INFO: secondmate sm-crashed relaunched after confirmed agent absence on existing endpoint (backend=tmux)
  BOOTSTRAP_INFO: secondmate sm-idle left down after confirmed agent absence on existing endpoint: no pending work (backend=tmux)
  BOOTSTRAP_INFO: secondmate sm-inflight relaunched after confirmed agent absence on existing endpoint (backend=tmux)
  BOOTSTRAP_INFO: secondmate sm-queued relaunched after confirmed agent absence on existing endpoint (backend=tmux)

================================================================
 WHAT THE FLEET ACTUALLY LAUNCHED
================================================================
  tmux calls made by the startup sweep:
    kill-window kill-window -t =firstmate:=fm-sm-crashed
    new-window new-window -dP -F #{window_id} -t firstmate: -n fm-sm-crashed -c <world>/sm-crashed
    kill-window kill-window -t =firstmate:=fm-sm-idle
    kill-window kill-window -t =firstmate:=fm-sm-inflight
    new-window new-window -dP -F #{window_id} -t firstmate: -n fm-sm-inflight -c <world>/sm-inflight
    kill-window kill-window -t =firstmate:=fm-sm-queued
    new-window new-window -dP -F #{window_id} -t firstmate: -n fm-sm-queued -c <world>/sm-queued

  tmux windows after startup:
    main claude
    fm-sm-crashed claude
    fm-sm-inflight claude
    fm-sm-queued claude

  verdict:
    sm-idle        left down
    sm-queued      RELAUNCHED
    sm-inflight    RELAUNCHED
    sm-crashed     RELAUNCHED
Evidence: Reproducer: end-to-end fleet startup demo script
#!/usr/bin/env bash
# Manual end-to-end demo of the fleet-startup launch policy.
# Builds a fake fleet whose tmux session died (all secondmate agent panes are
# bare shells / gone), then runs the real bin/fm-bootstrap.sh session-start
# sweep and shows which windows the fleet actually reopens.
set -u
ROOT=${FM_DEMO_ROOT:?}
W=$(mktemp -d "${TMPDIR:-/tmp}/fm-fleet-demo.XXXXXX")
export FM_GATE_REFUSE_BYPASS=1 FM_HOME_BINDING=test-harness
BASE_PATH=/usr/bin:/bin:/usr/sbin:/sbin
FB="$W/fakebin"; mkdir -p "$FB"
INV="$W/tmux-windows"; LOG="$W/tmux-calls.log"; : > "$LOG"

# --- fake tmux with a real window inventory ---------------------------------
cat > "$FB/tmux" <<'SH'
#!/usr/bin/env bash
set -u
inv=${FM_DEMO_INV:?}; log=${FM_DEMO_LOG:?}
win_of() { case "$1" in *:*) printf '%s' "${1##*:}" ;; *) printf '%s' "$1" ;; esac; }
strip() { printf '%s' "${1//=/}"; }
case "${1:-}" in
  display-message)
    tgt=""; for a in "$@"; do case "$a" in -t) t=1 ;; *) [ "${t:-}" = 1 ] && { tgt=$a; t=0; } ;; esac; done
    w=$(strip "$(win_of "$tgt")")
    for a in "$@"; do case "$a" in
      *pane_current_command*)
        grep -q "^$w " "$inv" 2>/dev/null || exit 1
        printf '%s\n' "$(grep "^$w " "$inv" | head -1 | cut -d' ' -f2)"; exit 0 ;;
      *pane_id*) printf '%%1\n'; exit 0 ;;
      *cursor_y*) printf '0\n'; exit 0 ;;
    esac; done
    exit 0 ;;
  list-windows) cut -d' ' -f1 "$inv" 2>/dev/null; exit 0 ;;
  new-window)
    printf 'new-window %s\n' "$*" >> "$log"
    for a in "$@"; do case "$a" in -n) n=1 ;; *) [ "${n:-}" = 1 ] && { printf '%s claude\n' "$(strip "$a")" >> "$inv"; n=0; } ;; esac; done
    exit 0 ;;
  kill-window)
    printf 'kill-window %s\n' "$*" >> "$log"
    tgt=""; for a in "$@"; do case "$a" in -t) t=1 ;; *) [ "${t:-}" = 1 ] && { tgt=$a; t=0; } ;; esac; done
    w=$(strip "$(win_of "$tgt")")
    grep -v "^$w " "$inv" > "$inv.n" 2>/dev/null; mv "$inv.n" "$inv"; exit 0 ;;
  has-session) exit 0 ;;
esac
exit 0
SH
chmod +x "$FB/tmux"
for b in node gh gh-axi chrome-devtools-axi lavish-axi pi-signed quota-axi; do
  printf '#!/usr/bin/env bash\nexit 0\n' > "$FB/$b"; chmod +x "$FB/$b"
done
printf '#!/usr/bin/env bash\n[ "${1:-}" = get ] && [ "${2:-}" = --help ] && printf "Usage: treehouse get [--lease]\\n"\nexit 0\n' > "$FB/treehouse"
printf '#!/usr/bin/env bash\n[ "${1:-}" = --version ] && printf "no-mistakes version v1.31.2 (fake)\\n"\nexit 0\n' > "$FB/no-mistakes"
cat > "$FB/tasks-axi" <<'SH'
#!/usr/bin/env bash
case "${1:-} ${2:-}" in
  "--version ") printf '0.1.1\n' ;;
  "update --help") printf 'usage: tasks-axi update <id> [flags]\n  --archive-body\n' ;;
  "mv --help") printf 'usage: tasks-axi mv <id> [<id>...] --to <path-or-dir>\n' ;;
esac
exit 0
SH
chmod +x "$FB/treehouse" "$FB/no-mistakes" "$FB/tasks-axi"

# --- the firstmate home -----------------------------------------------------
mkdir -p "$W/home/state" "$W/home/config"
touch "$W/home/state/.last-watcher-beat"
printf 'claude\n' > "$W/home/config/crew-harness"

# seed_mate <id> <backlog-body> [child-meta]
seed_mate() {
  local id=$1 body=$2 child=${3:-}
  local home="$W/$id"
  mkdir -p "$home/bin" "$home/data" "$home/state" "$home/config" "$home/projects"
  printf '%s\n' "$id" > "$home/.fm-secondmate-home"
  printf '# Firstmate\n' > "$home/AGENTS.md"
  printf 'charter\n' > "$home/data/charter.md"
  printf '%s' "$body" > "$home/data/backlog.md"
  [ -n "$child" ] && printf 'kind=ship\nwindow=firstmate:fm-%s-child\n' "$id" > "$home/state/$child"
  { printf 'window=firstmate:fm-%s\n' "$id"; printf 'kind=secondmate\n'
    printf 'harness=claude\n'; printf 'home=%s\n' "$home"; } > "$W/home/state/$id.meta"
}

seed_mate sm-idle    '## In flight

## Queued

## Done
- [x] shipped-last-week - all done
'
seed_mate sm-queued  '## In flight

## Queued
- [ ] wire-up-report - captain queued this before the crash

## Done
'
seed_mate sm-inflight '## In flight
- [ ] migrate-db - paused mid-flight when the session hit its quota limit

## Queued

## Done
'
seed_mate sm-crashed '## In flight

## Queued

## Done
' child1.meta

# The session died: every secondmate window is now a bare shell (agent gone).
{ printf 'main claude\n'; printf 'fm-sm-idle zsh\n'; printf 'fm-sm-queued zsh\n'
  printf 'fm-sm-inflight zsh\n'; printf 'fm-sm-crashed zsh\n'; } > "$INV"

echo "================================================================"
echo " FLEET STATE AT STARTUP - four secondmates registered, all down"
echo "================================================================"
for id in sm-idle sm-queued sm-inflight sm-crashed; do
  printf '\n--- %s ---\n' "$id"
  printf '  data/backlog.md:\n'
  sed 's/^/    | /' "$W/$id/data/backlog.md"
  printf '  state/*.meta (child tasks it dispatched): '
  ls "$W/$id/state"/*.meta >/dev/null 2>&1 && ls -1 "$W/$id/state"/*.meta | xargs -n1 basename | tr '\n' ' ' || printf '(none)'
  printf '\n'
done

echo
echo "  tmux windows before startup (agent pane command per window):"
sed 's/^/    /' "$INV"

echo
echo "================================================================"
echo " RUNNING FLEET STARTUP: bin/fm-bootstrap.sh"
echo "================================================================"
DIGEST="$W/digest.txt"
PATH="$FB:$BASE_PATH" TMUX='' FM_BACKEND=tmux FM_HOME="$W/home" \
  FM_DEMO_INV="$INV" FM_DEMO_LOG="$LOG" FM_SEND_SETTLE=0 \
  env ${FM_DEMO_VERBOSE:+FM_BOOTSTRAP_VERBOSE_FACTS=1} \
  "$ROOT/bin/fm-bootstrap.sh" > "$DIGEST" 2>&1
if [ -n "${FM_DEMO_VERBOSE:-}" ]; then
  echo "  (FM_BOOTSTRAP_VERBOSE_FACTS=1 - showing the per-secondmate facts)"
  grep -E 'SECONDMATE_LIVENESS|BOOTSTRAP_INFO: secondmate|NUDGE_SECONDMATES|CONFIG_REREAD' \
    "$DIGEST" | sed 's/^/  /'
else
  echo "  (default verbosity - every actionable secondmate line the captain sees)"
  if grep -qE 'SECONDMATE_LIVENESS|NUDGE_SECONDMATES|CONFIG_REREAD|BOOTSTRAP_INFO: nudged' "$DIGEST"; then
    grep -E 'SECONDMATE_LIVENESS|NUDGE_SECONDMATES|CONFIG_REREAD|BOOTSTRAP_INFO: nudged' \
      "$DIGEST" | sed 's/^/  /'
  else
    echo "    <none - the correct outcome is silent; no actionable diagnostic>"
  fi
fi

echo
echo "================================================================"
echo " WHAT THE FLEET ACTUALLY LAUNCHED"
echo "================================================================"
echo "  tmux calls made by the startup sweep:"
sed -e "s#$W#<world>#g" -e 's/^/    /' "$LOG"
echo
echo "  tmux windows after startup:"
sed 's/^/    /' "$INV"
echo
echo "  verdict:"
for id in sm-idle sm-queued sm-inflight sm-crashed; do
  if grep -q "new-window.*fm-$id\b" "$LOG"; then
    printf '    %-14s RELAUNCHED\n' "$id"
  else
    printf '    %-14s left down\n' "$id"
  fi
done
rm -rf "$W"
Evidence: Key excerpt — what fleet startup actually launched
FLEET STATE AT STARTUP - four secondmates registered, all down

--- sm-idle --- backlog: ## Queued (empty), ## In flight (empty) child metas: (none)
--- sm-queued --- backlog: ## Queued -> wire-up-report child metas: (none)
--- sm-inflight --- backlog: ## In flight -> migrate-db (quota limit) child metas: (none)
--- sm-crashed --- backlog: (all empty) child metas: child1.meta

tmux windows before startup (agent pane command per window):
main claude
fm-sm-idle zsh
fm-sm-queued zsh
fm-sm-inflight zsh
fm-sm-crashed zsh

RUNNING FLEET STARTUP: bin/fm-bootstrap.sh
(default verbosity - every actionable secondmate line the captain sees)
<none - the correct outcome is silent; no actionable diagnostic>

tmux calls made by the startup sweep:
kill-window -t =firstmate:=fm-sm-crashed
new-window -n fm-sm-crashed -c <world>/sm-crashed
kill-window -t =firstmate:=fm-sm-idle <-- husk cleaned up, NO relaunch
kill-window -t =firstmate:=fm-sm-inflight
new-window -n fm-sm-inflight -c <world>/sm-inflight
kill-window -t =firstmate:=fm-sm-queued
new-window -n fm-sm-queued -c <world>/sm-queued

tmux windows after startup:
main claude
fm-sm-crashed claude
fm-sm-inflight claude
fm-sm-queued claude

verdict:
sm-idle left down
sm-queued RELAUNCHED
sm-inflight RELAUNCHED
sm-crashed RELAUNCHED

--- same fleet under FM_BOOTSTRAP_VERBOSE_FACTS=1 ---
BOOTSTRAP_INFO: secondmate sm-crashed relaunched after confirmed agent absence on existing endpoint (backend=tmux)
BOOTSTRAP_INFO: secondmate sm-idle left down after confirmed agent absence on existing endpoint: no pending work (backend=tmux)
BOOTSTRAP_INFO: secondmate sm-inflight relaunched after confirmed agent absence on existing endpoint (backend=tmux)
BOOTSTRAP_INFO: secondmate sm-queued relaunched after confirmed agent absence on existing endpoint (backend=tmux)
- Outcome: ⚠️ 1 info across 1 run (7m11s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 3 issues found → auto-fixed (2) ✅
  • ⚠️ bin/fm-bootstrap.sh:555 - A secondmate deliberately left down for having no pending work keeps its kind=secondmate meta, and the sweeps that run after it treat meta presence as liveness (bin/fm-ff-lib.sh:247 states this explicitly; process_secondmate at bin/fm-ff-lib.sh:390 never probes). Concrete path: secondmate_liveness_sweep (bin/fm-bootstrap.sh:994) leaves idle sm1 down silently; secondmate_sync (:995) fast-forwards its home, gets FF_STATUS=updated with a non-empty FF_INSTR, and calls fm_ff_after_instruction_update -> secondmate_send_nudge, which runs fm-send.sh fm-sm1 against the dead window. The send fails, printing an actionable 'NUDGE_SECONDMATES: secondmate sm1: send failed: ...' line and leaving a persistent state/.secondmate-nudge-pending/sm1.pending marker that secondmate_retry_pending_nudges re-fails every session. The config-inheritance loop at bin/fm-bootstrap.sh:407-455 does the same via fm_config_send_reread_nudge, and because sm1 is absent from SECONDMATE_RESPAWNED_IDS, reread_skip_pending stays 0 so pending reread generations accumulate until 'CONFIG_REREAD: secondmate sm1: send failed: retry instruction queue is full'. Before this change this was unreachable: the liveness sweep respawned every dead mate at :994, so the endpoint was live by :995. The intent states the correctly-left-down-idle case must not produce an actionable diagnostic, and bootstrap-diagnostics tells the agent to fix the endpoint and rerun session start - now unfixable by design. Earliest shared boundary: have the liveness sweep record the ids it deliberately left down (mirroring SECONDMATE_RESPAWNED_IDS) and have the sync and inherit sweeps skip only the send for those ids, still fast-forwarding the home and propagating files.
  • ℹ️ bin/fm-bootstrap.sh:547 - The new pending-work gate uses the recorded home= path directly, unlike every other consumer of that field (process_secondmate at bin/fm-ff-lib.sh:397 and secondmate_retry_pending_nudges at bin/fm-bootstrap.sh:360-364 both run validate_secondmate_home, which requires the .fm-secondmate-home marker). A stale home= pointing at the primary checkout passes [ -d ] and makes the state/*.meta glob match the primary's own secondmate metas, so the mate is relaunched every session as 'pending work' - the idle relaunch the policy forbids; a home= pointing at any other existing directory reads as idle and strands a busy mate. Routing the resolution through validate_secondmate_home would turn both into the 'cannot judge pending work' skip this change deliberately added.
  • ℹ️ bin/fm-bootstrap.sh:484 - secondmate_home_has_pending_work cannot distinguish awk's 'no items' exit 1 from an awk error (exit 2) on an existing but unreadable or malformed data/backlog.md; both fall through to the state/ glob and then to 'no pending work'. A home with real '## In flight' items but an unreadable backlog and no dispatched children is left down silently, with no SECONDMATE_LIVENESS line - inconsistent with the refuse-to-guess skips this same change added for an unresolvable home. awk's stderr is also unredirected, so the raw error lands in the bootstrap digest.

🔧 Fix: stop nudging left-down mates; validate home and backlog reads
1 warning still open:

  • ⚠️ bin/fm-bootstrap.sh:356 - Skipping the retry for a left-down mate preserves its nudge marker, but the home keeps fast-forwarding while the mate is down and the marker's commit is never refreshed, so the reopening session fails the commit guard permanently. Concrete path: session A, sm1 live, instructions advance, home ff's to c1, fm-send.sh fails transiently, marker written with commit=c1. Session B, sm1's endpoint is dead and its backlog empty, so the liveness sweep leaves it down and adds it to SECONDMATE_LEFT_DOWN_IDS; secondmate_retry_pending_nudges skips it at :356 keeping commit=c1, while process_secondmate still fast-forwards the home to c2 and fm_ff_after_instruction_update returns early at :340, so the marker is never rewritten. Session C, after the captain reopens sm1: it reads alive, is no longer left down, the retry runs, head (c2) != commit (c1) hits the guard at :393, and 'NUDGE_SECONDMATES: secondmate sm1: send failed: retry target is not at recorded instruction commit' prints; the marker is only removed on a successful send, so the actionable line recurs every session forever. This is the same unfixable-diagnostic class this commit set out to remove, and bootstrap-diagnostics still instructs the operator to keep the marker and rerun session start. Minimal fix, consistent with the reasoning this same commit already applies to reread_skip_pending: drop the marker when the mate is left down, because the launch that eventually reopens it re-reads its instructions at startup anyway, and correct the comment at :353-355 which currently promises the marker is kept 'for the session that reopens it'.

🔧 Fix: retire unsatisfiable nudge markers for left-down secondmates
✅ Re-checked - no issues remain.

⚠️ **Test** - 1 info
  • ℹ️ tests/fm-bootstrap.test.sh:403 - Pre-existing, environment-dependent failure in tests/fm-bootstrap.test.sh: test_orca_backend_gates_orca_tool_only_when_selected asserts bootstrap reports MISSING: orca, but this host ships GNOME's screen reader at /usr/bin/orca, which is inside the test's BASE_PATH (/usr/bin:/bin:/usr/sbin:/sbin), so the tool resolves as present and the expected line is never emitted. Reproduced identically from a clean git archive of base commit c2e506b, so it predates this branch and is unrelated to the secondmate launch-policy change. Not fixed here because the repair belongs to an unrelated test and would add noise to this diff.
  • bash tests/fm-secondmate-liveness.test.sh — all 21 cases pass, including the six new ones: idle mate left down, in-flight child metadata relaunches, and the no-home / missing-home / unsafe-home / unreadable-backlog refuse-to-guess skips
  • bash tests/fm-secondmate-sync.test.sh — all 22 cases pass, including T8g (left-down home still fast-forwards but is never sent into) and T8h (unsatisfiable nudge marker retired rather than carried into the reopening session)
  • Manual end-to-end fleet startup: /tmp/no-mistakes-evidence/01KZKBZ27D9GHDDYP9QP15RQE9/fleet-startup-demo.sh builds a firstmate home with four registered secondmates (sm-idle empty backlog, sm-queued item under ## Queued, sm-inflight item under ## In flight, sm-crashed empty backlog + a leftover state/child1.meta), kills every agent pane to bare shells, then runs the real bin/fm-bootstrap.sh against a fake tmux carrying a live window inventory
  • Same demo re-run with FM_BOOTSTRAP_VERBOSE_FACTS=1 to confirm the silent left-down case surfaces as a BOOTSTRAP_INFO fact and nothing more
  • Default-verbosity digest grepped for SECONDMATE_LIVENESS|NUDGE_SECONDMATES|CONFIG_REREAD|BOOTSTRAP_INFO: nudged — zero matches, confirming the correct left-down outcome emits no actionable diagnostic
  • grep -rn &#39;## In flight|## Queued&#39; bin/fm-backlog-handoff.sh bin/fm-session-start.sh bin/fm-fleet-snapshot.sh — confirms the headings secondmate_home_has_pending_work parses are byte-identical to the ones seeded at bin/fm-backlog-handoff.sh:318
  • bash tests/fm-bootstrap.test.sh — direct suite for the changed script; one pre-existing orca failure (see findings), all other cases pass
  • bash tests/fm-documentation-audiences.test.sh — all 4 cases pass, covering the AGENTS.md / docs/ / skill prose surfaces this branch edits
  • Base-commit control: git archive c2e506b | tar -x -C /tmp/fm-base-check &amp;&amp; bash tests/fm-bootstrap.test.sh — reproduces the identical orca failure, proving it predates the branch
⚠️ **Document** - 1 info
  • ℹ️ docs/configuration.md:316 - docs/configuration.md:316 and .agents/skills/secondmate-provisioning/SKILL.md:114-115 still describe bin/fm-config-push.sh as using "live secondmate discovery", but that discovery is the same meta-record-based helper the bootstrap sweep uses, so a mid-session config push would target a secondmate the liveness sweep deliberately left down (whose endpoint is dead by design) and report a send error. Left unchanged deliberately: the mid-session push path is not touched by this branch, and whether it should skip left-down mates is a behavior decision, not a doc fix. Worth reconciling in the follow-up that also resolves the 'secondmate-no-autoload' backlog item.
✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Bchue added 7 commits August 9, 2026 01:00
…istration

Fleet startup previously relaunched every registered dead/missing secondmate
unconditionally. The captain decided 2026-08-03 (superseding the 2026-07-31
open-at-the-stop rule) that startup should launch only the primary, and
relaunch a secondmate only when its own durable records show pending work
(a queued/in-flight backlog item, or an undelivered in-flight child task),
since an idle secondmate just burns startup tokens with no work to do.
…ing-work rule

The diagnostic handling playbook still framed the sweep as guaranteeing every
registered secondmate is live, which now reads as a contradiction: a registered
secondmate with no pending work is deliberately left down and stays silent. It
also did not cover the two new skip reasons that refuse to guess pending work
from an unresolvable home.
secondmate_home_has_pending_work bound its home argument to a local it never
read, since both derived paths build off "$1" directly.
@brchue-ux
brchue-ux merged commit 4e0ddcd into main Aug 9, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant