fix(ensemble-workflow): null-skip fail-open in shared harness + harness regression tests

.claude-plugin/marketplace.json

@@ -9,7 +9,7 @@
       "name": "parallel-ai-agents",
       "source": "./plugins/parallel-ai-agents",
       "description": "平行派發任務給多個 AI agent（Claude + Codex），獨立執行後交叉比對結果。Codex 改走直接 HTTP wrapper（bin/codex-call，Swift script）取代 codex exec subprocess，解決 hang 問題且避開 Python 版本飄移",
-      "version": "2.14.0",
+      "version": "2.14.1",
       "author": {
         "name": "Che Cheng"
       },

.github/workflows/test.yml

@@ -27,3 +27,6 @@ jobs:
 
       - name: bats
         run: bats test/
+
+      - name: harness regression (node)
+        run: node test/ensemble-workflow.test.mjs

plugins/parallel-ai-agents/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "parallel-ai-agents",
   "description": "平行派發任務給多個 AI agent（Claude + Codex），獨立執行後交叉比對結果。Codex 改走直接 HTTP wrapper（bin/codex-call，Swift script）取代 codex exec subprocess，解決 hang 問題且避開 Python 版本飄移",
-  "version": "2.14.0",
+  "version": "2.14.1",
   "author": {
     "name": "Che Cheng"
   }

plugins/parallel-ai-agents/CHANGELOG.md

@@ -11,6 +11,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [2.14.1] - 2026-06-03
+
+### Fixed
+- **共用 harness `ensemble-workflow.js` 的 null-skip fail-OPEN**（dogfood lecture/academic/compose/harness 時抓到）：Workflow runtime 在「使用者中途 skip 某 agent」時讓 `agent()` 回 `null`。但 review / codex / devil's-advocate 三個 `.then` 只用 `(r && r.findings) || []` 處理 null 的 findings、**沒處理 `ok` flag** → 被 skip 的 reviewer 被當 `ok:true`（乾淨通過），**繞過 fail-closed integrity 檢查 → 可能假 PASS**（與 code-review 三輪一直在防的「假綠燈」同類，只是換成 JS null 路徑）。修：三處 `.then` 把 `r == null` 視為 `ok:false` → core lens／DA 被 skip 會如預期觸發 HIGH integrity finding。
+
+### Added
+- **`test/ensemble-workflow.test.mjs` —— 共用 harness 的 node regression 測試**（8 個）：unknown profile、空 lens 組合、core lens 被 skip(null)/error(throw)/DA 缺席 → HIGH integrity（鎖死上述 fail-open 修正）、codex 缺席 → INFO 非阻塞、mergeDedup 對 malformed severity 穩健。把 workflow script body 包成可 import 的 async 函式、注入 mock globals 實跑。已接進 `test/run.sh` 與 CI。
+
+### Docs
+- **plugin `CLAUDE.md` 修正 drift**：原本只列已不存在的單一 `/parallel-ai-agents:ensemble-review` + 「4 teammates + 1 Codex」舊架構；改成實際的 4 個 skill（code/academic/lecture/compose）+ 雙 backend（Workflow harness 預設、legacy fallback）+ fail-closed 說明。
+
 ## [2.14.0] - 2026-06-03
 
 ### Added

plugins/parallel-ai-agents/CLAUDE.md

@@ -6,22 +6,31 @@
 
 ## Skills
 
-| Skill | 用途 | 架構 |
-|-------|------|------|
-| `/parallel-ai-agents:ensemble-review` | 審閱文件/程式碼，交叉比對產出共識/盲點報告 | 4 Claude teammates (team) + 1 Codex |
+| Skill | 用途 |
+|-------|------|
+| `/parallel-ai-agents:ensemble-code-review` | 審程式碼/技術文件/diff（最完整：`--diff`/`--base`/`--since`/`--commits`/`--pr` 等 diff 模式，建構走 `bin/pai-build-diff`）|
+| `/parallel-ai-agents:ensemble-academic-review` | 審學術論文（methodology/writing + 文獻真偽 + 數字重算；支援多輪 mix/hybrid/auto-iterate）|
+| `/parallel-ai-agents:ensemble-lecture-review` | 審教學講義（內容正確性/可讀性/逐字稿覆蓋率）|
+| `/parallel-ai-agents:ensemble-compose` | 自由組合：跨 profile 挑 lens + 自訂 reviewer（`--include`/`--lens`/`--lens-file` CSV）|
 
-## 審閱架構
+四個 skill 共用同一個 harness `workflows/ensemble-workflow.js`（`PROFILES` 內建各角色 lens；regression 測試見 `test/`）。
+
+## 審閱架構（雙 backend，findings 形狀一致）
+
+預設 **Backend A — Workflow harness**（`workflows/ensemble-workflow.js`）：distinct-lens reviewers（× `replicas`）+ devil's-advocate（讀同儕**完稿**反駁）+ 可選 Codex 跨模型盲驗 → pure-JS mergeDedup + **fail-closed**（core lens／DA 缺席 = HIGH integrity，不可假 PASS）。reviewer 數是資料（lens 陣列 × replicas），可大量 fan-out（硬上限 `MAX_AGENTS`）。
 
 ```
-ensemble-review
-├── Claude Team（4 teammates，orchestrated）
+ensemble（以 code profile 為例）
+├── reviewers（× replicas，平行 barrier）
 │   ├── architecture — 設計、API、依賴
 │   ├── correctness — 邏輯、bug、edge case
-│   ├── security — 攻擊者視角
-│   └── devils-advocate — 反駁前 3 人
-└── Codex（gpt-5.5，跨模型盲驗）
+│   └── security — 攻擊者視角
+├── devil's-advocate — 讀上面完稿、反駁「通過/LOW」（downstream node，非 live SendMessage）
+└── Codex（gpt-5.5，跨模型盲驗，--codex 開）
 ```
 
+`Workflow` tool 不存在的舊版 Claude Code → fallback **Backend B — legacy TeamCreate fan-out**（同一 message 啟動 N 個 general-purpose Agent + Codex Bash）。兩 backend 報表一致。
+
 ## 依賴
 
 - Claude Code orchestrated teams（TeamCreate、SendMessage）

plugins/parallel-ai-agents/test/README.md

@@ -7,8 +7,11 @@
 | 檔案 | 對象 |
 |------|------|
 | `pai-build-diff.bats` | `../bin/pai-build-diff`（diff 模式建構器，3 輪硬化的單一真相源）|
+| `ensemble-workflow.test.mjs` | `../workflows/ensemble-workflow.js`（共用 harness，4 個 skill 的底層）|
 
-涵蓋：5 種模式（`--diff`/`--base`/`--since`/`--commits`/`--pr`）、退出碼契約（0 有 diff／3 無變更／1 錯誤）、ref/N 驗證（injection、dashed-ref、0/leading-zero、位數溢位）、untracked 安全（symlink no-follow、FIFO no-hang、換行檔名 C-quote）、empty-tree base、未知 mode 的多位元組 regression。
+`pai-build-diff.bats` 涵蓋：5 種模式（`--diff`/`--base`/`--since`/`--commits`/`--pr`）、退出碼契約（0 有 diff／3 無變更／1 錯誤）、ref/N 驗證（injection、dashed-ref、0/leading-zero、位數溢位）、untracked 安全（symlink no-follow、FIFO no-hang、換行檔名 C-quote）、empty-tree base、未知 mode 的多位元組 regression。
+
+`ensemble-workflow.test.mjs` 涵蓋 harness 的 **fail-closed 不變式**：unknown profile、空 lens 組合、core lens 被 skip（null）/ error（throw）/ devil's-advocate 缺席 → 一律 HIGH integrity（不可假 PASS）、codex 缺席 → INFO 非阻塞、mergeDedup 對 malformed severity 穩健。把「null-skip fail-open」的修正鎖死成 regression。（純 node，無框架；把 workflow script body 包成可 import 的 async 函式、注入 mock globals 實跑。）
 
 ## 跑法
 

plugins/parallel-ai-agents/test/ensemble-workflow.test.mjs

@@ -0,0 +1,117 @@
+#!/usr/bin/env node
+// ensemble-workflow.js 的 regression 測試（純 node，無框架）。
+//
+// 跑法：node test/ensemble-workflow.test.mjs
+//
+// harness 是 Workflow script（top-level await/return），無法被標準 runner 直接 import。
+// 解法：讀原始碼 → 把 body 包成一個可 export 的 async 函式（注入 mock 的
+// agent/parallel/phase/log/args 當參數）→ 寫成 temp module 後 import → 實跑整個
+// orchestration 並斷言。鎖死 fail-closed 不變式（含 null-skip fail-open 的修正）。
+// 載入的是本 repo 自己的 source（被測碼本身），信任等級等同 import 任何模組。
+
+import { readFileSync, writeFileSync, rmSync } from 'node:fs'
+import { fileURLToPath, pathToFileURL } from 'node:url'
+import { dirname, join } from 'node:path'
+import { tmpdir } from 'node:os'
+import assert from 'node:assert/strict'
+
+const __dirname = dirname(fileURLToPath(import.meta.url))
+const harnessSrc = readFileSync(join(__dirname, '..', 'workflows', 'ensemble-workflow.js'), 'utf8')
+  .replace(/^export const meta/m, 'const meta') // strip ESM export so the body is legal inside a function
+
+// Wrap the workflow-script body as an exportable async fn whose destructured params ARE the workflow
+// globals the harness references (agent/parallel/pipeline/phase/log/args). Write → import (no eval).
+const wrapped = `export async function runEnsemble({ agent, parallel, pipeline, phase, log, args }) {\n${harnessSrc}\n}\n`
+const tmpFile = join(tmpdir(), `pai-ensemble-harness-${process.pid}.mjs`)
+writeFileSync(tmpFile, wrapped)
+let rawRun
+try {
+  ;({ runEnsemble: rawRun } = await import(pathToFileURL(tmpFile).href))
+} finally {
+  rmSync(tmpFile, { force: true })
+}
+
+const parallel = async (thunks) => Promise.all(thunks.map((f) => f()))
+const pipeline = async () => []
+const noop = () => {}
+
+// run one scenario: agentImpl(prompt, opts) decides per-label behavior (return obj / null=skip / throw).
+function runEnsemble(args, agentImpl) {
+  return rawRun({ agent: agentImpl, parallel, pipeline, phase: noop, log: noop, args })
+}
+
+// agent impls keyed by label
+const allPass = async () => ({ findings: [] })
+const skip = (label) => async (_p, o) => (o && o.label === label ? null : { findings: [] })
+const boom = (label) => async (_p, o) => { if (o && o.label === label) throw new Error('boom'); return { findings: [] } }
+
+const tests = []
+const test = (name, fn) => tests.push({ name, fn })
+
+test('unknown profile → HIGH harness finding, agents=0', async () => {
+  const out = await runEnsemble({ profile: 'nope', file: '/x' }, allPass)
+  assert.equal(out.verdict, 'FINDINGS')
+  assert.equal(out.stats.agents, 0)
+  assert.match(out.findings[0].title, /unknown ensemble profile/)
+})
+
+test('custom profile, no lenses → HIGH "no active lenses"', async () => {
+  const out = await runEnsemble({ profile: 'custom', file: '/x' }, allPass)
+  assert.equal(out.verdict, 'FINDINGS')
+  assert.ok(out.findings.some((f) => /no active lenses/.test(f.title) && f.severity === 'HIGH'))
+})
+
+test('code profile all-ok → PASS, no integrity', async () => {
+  const out = await runEnsemble({ profile: 'code', file: '/x', codexEnabled: false }, allPass)
+  assert.equal(out.verdict, 'PASS')
+  assert.equal(out.stats.integrity, 0)
+})
+
+test('REGRESSION: skip a core lens (null) → fail-closed HIGH (not silent PASS)', async () => {
+  const out = await runEnsemble({ profile: 'code', file: '/x', codexEnabled: false }, skip('review:security'))
+  assert.equal(out.verdict, 'FINDINGS')
+  assert.ok(out.findings.some((f) => f.lens === 'security' && f.severity === 'HIGH' && /did not complete/.test(f.title)))
+})
+
+test('error (throw) a core lens → fail-closed HIGH', async () => {
+  const out = await runEnsemble({ profile: 'code', file: '/x', codexEnabled: false }, boom('review:correctness'))
+  assert.equal(out.verdict, 'FINDINGS')
+  assert.ok(out.findings.some((f) => f.lens === 'correctness' && f.severity === 'HIGH' && /did not complete/.test(f.title)))
+})
+
+test('skip devils-advocate → fail-closed HIGH', async () => {
+  const out = await runEnsemble({ profile: 'code', file: '/x', codexEnabled: false }, skip('devils-advocate'))
+  assert.equal(out.verdict, 'FINDINGS')
+  assert.ok(out.findings.some((f) => f.lens === 'devils-advocate' && f.severity === 'HIGH'))
+})
+
+test('skip codex (codexEnabled) → INFO process gap, verdict stays PASS', async () => {
+  const out = await runEnsemble({ profile: 'code', file: '/x', codexEnabled: true, codexCallPath: '/bin/codex-call' }, skip('codex'))
+  assert.equal(out.verdict, 'PASS') // codex gap is non-blocking
+  assert.ok(out.findings.some((f) => f.lens === 'codex' && f.severity === 'INFO'))
+})
+
+test('mergeDedup robust to malformed severity (no crash, finding kept, FINDINGS)', async () => {
+  const weird = async (_p, o) =>
+    o && o.label === 'review:architecture'
+      ? { findings: [{ severity: 'BOGUS', title: 'odd', body: 'b', file: null }] }
+      : { findings: [] }
+  const out = await runEnsemble({ profile: 'code', file: '/x', codexEnabled: false }, weird)
+  assert.ok(out.findings.some((f) => f.title === 'odd')) // unknown severity ranked 0, not dropped/crashed
+  assert.equal(out.verdict, 'FINDINGS') // 'BOGUS' !== 'INFO'
+})
+
+let pass = 0
+let fail = 0
+for (const t of tests) {
+  try {
+    await t.fn()
+    console.log(`ok - ${t.name}`)
+    pass++
+  } catch (e) {
+    console.log(`not ok - ${t.name}\n  ${e.message}`)
+    fail++
+  }
+}
+console.log(`\n${pass}/${tests.length} passed`)
+process.exit(fail === 0 ? 0 : 1)

plugins/parallel-ai-agents/test/run.sh

@@ -10,4 +10,7 @@ shellcheck bin/pai-build-diff
 echo "── bats test/ ──"
 bats test/
 
+echo "── harness regression (node) ──"
+node test/ensemble-workflow.test.mjs
+
 echo "✓ 全部通過"

plugins/parallel-ai-agents/workflows/ensemble-workflow.js

@@ -437,9 +437,12 @@ if (replicas < requested) {
 
 // Phase 1 (barrier): every base lens × replicas + optional Codex run concurrently and
 // independently. A barrier is correct: the devil's-advocate (phase 2) needs every reviewer's
-// findings to refute them. Each thunk CATCHES its own error and tags its lens (ok:false) so a
-// failed lens is observable downstream — never silently dropped. Each finding's lens is forced to
-// the assigned key so an agent cannot mislabel its attribution.
+// findings to refute them. Each thunk CATCHES its own error AND treats a null agent() return
+// (the runtime hands back null when the user SKIPS an agent mid-run) as ok:false — both are
+// "this lens did not actually review", so both must surface as a fail-closed integrity finding
+// downstream, never a silent pass. (A null slipping through as ok:true with empty findings was a
+// fail-OPEN false-green-light, caught by this harness's own dogfood.) Each finding's lens is forced
+// to the assigned key so an agent cannot mislabel its attribution.
 phase('review')
 const reviewThunks = []
 for (const l of activeLenses) {
@@ -450,15 +453,19 @@ for (const l of activeLenses) {
         label: replicas > 1 ? `review:${l.key}#${k + 1}` : `review:${l.key}`,
         phase: 'review',
       })
-        .then((r) => ({ lens: l.key, findings: ((r && r.findings) || []).map((f) => ({ ...f, lens: l.key })), ok: true }))
+        .then((r) => (r == null
+          ? { lens: l.key, findings: [], ok: false }                                   // user-skipped → fail-closed
+          : { lens: l.key, findings: (r.findings || []).map((f) => ({ ...f, lens: l.key })), ok: true }))
         .catch(() => ({ lens: l.key, findings: [], ok: false }))
     )
   }
 }
 const codexThunk = codexOn
   ? () =>
       agent(codexPrompt(profile, A), { schema: FINDINGS_SCHEMA, label: 'codex', phase: 'review' })
-        .then((r) => ({ lens: 'codex', findings: ((r && r.findings) || []).map((f) => ({ ...f, lens: 'codex' })), ok: true }))
+        .then((r) => (r == null
+          ? { lens: 'codex', findings: [], ok: false }                                 // user-skipped → surfaced as process gap
+          : { lens: 'codex', findings: (r.findings || []).map((f) => ({ ...f, lens: 'codex' })), ok: true }))
         .catch(() => ({ lens: 'codex', findings: [], ok: false }))
   : null
 
@@ -468,7 +475,9 @@ const reviewerResults = round1.filter((r) => r.lens !== 'codex')
 // Phase 2: devil's-advocate adversarially refutes the reviewers' judgments (also fail-aware).
 phase('adversarial')
 const da = await agent(daPrompt(profile, reviewerResults, A), { schema: FINDINGS_SCHEMA, label: 'devils-advocate', phase: 'adversarial' })
-  .then((r) => ({ findings: ((r && r.findings) || []).map((f) => ({ ...f, lens: 'devils-advocate' })), ok: true }))
+  .then((r) => (r == null
+    ? { findings: [], ok: false }                                                      // user-skipped DA → fail-closed
+    : { findings: (r.findings || []).map((f) => ({ ...f, lens: 'devils-advocate' })), ok: true }))
   .catch(() => ({ findings: [], ok: false }))
 
 // Phase 3: merge + dedup (pure JS; no agent, no FS). FAIL-CLOSED — a core lens (every base lens