feat(brain): FP-discipline triage model + no-silent-substitution safety

.gitignore

@@ -131,3 +131,14 @@ logs/
 
 # IDE/agent local config (not part of the tool)
 .antigravitycli/
+
+# ── Operator review-scratch (never commit — codex/grok/agy output may name clients) ─
+.friends_*/
+
+# ── Cloned third-party tools (external repos, not part of Vikramaditya) ────────
+tools/LinkFinder/
+tools/SecretFinder/
+tools/XSStrike/
+tools/graphql-cop/
+tools/nomore403/
+tools/__pycache__/

README.md

@@ -124,12 +124,21 @@ python3 vikramaditya.py example.com            # auto-detect, interactive
 For autonomous operation, install [Ollama](https://ollama.com) and pull the per-role models the brain actually uses (see [AI Brain & Models](#ai-brain--models) for the rationale):
 
 ```bash
-ollama pull phi4:14b               # faithful narrator (default since v9.23)
-ollama pull devstral-small-2:24b   # primary exploit coder (A/B-validated)
-ollama pull qwen2.5-coder:14b      # fast fallback coder
+# Scanner / exploit code-gen — pullable from the Ollama library:
+ollama pull qwen3-coder:30b        # lands grounded PoCs (or qwen2.5-coder:14b for a lighter box)
+
+# Triage + narrator — the 2026-07-16 FP-discipline bench winner is OpenMythos-27B, a GGUF IMPORT
+# (NOT in the Ollama library). Build it once, or use a pullable faithful generic as fallback:
+#   download hf.co/jabbatheduck/OpenMythos-GGUF (Q4_K) → ollama create openmythos-27b -f Modelfile
+ollama pull qwen3:14b              # pullable faithful narrator/triage fallback if you skip the import
 ```
 
-> **Minimum for autonomous mode:** `phi4:14b` (narrator) **plus one** coder — either `devstral-small-2:24b` *or* `qwen2.5-coder:14b` (the brain auto-prefers Devstral when both are present). These three are each multi-GB downloads.
+> **Minimum for autonomous mode:** one triage/narrator model — `openmythos-27b` (the bench winner, a
+> GGUF import) *or* the pullable `qwen3:14b` fallback — **plus one** coder (`qwen3-coder:30b` or
+> `qwen2.5-coder:14b`). See [AI Brain & Models](#ai-brain--models) and the benchmark record at
+> [`docs/benchmarks/2026-07-16-triage-fp-discipline.md`](docs/benchmarks/2026-07-16-triage-fp-discipline.md).
+> A pinned model that isn't installed now warns loudly (and fails under `BRAIN_REQUIRE_PIN=1`), so pin
+> only models you've actually pulled/built.
 
 Optionally create the security-tuned triage model (fully optional — triage defaults to `phi4:14b` without it):
 
@@ -278,16 +287,21 @@ role wants a different model — these are env-overridable with **no code change
 
 | Role | Model | Why | Env var |
 |:--|:--|:--|:--|
-| **Narration / analysis** | `phi4:14b` | Lowest hallucination of any local model (Vectara 3.7 %) — won't fabricate findings | `BRAIN_MODEL=phi4:14b` |
-| **Triage** (submit/drop) | `phi4:14b` *(default)* → `bugtraceai-apex` *(opt-in)* | Triage defaults to `phi4:14b` for speed and consistent JSON. Pull `bugtraceai-apex` and set `TRIAGE_MODEL` to switch to the security-DPO judge (empirically beat phi4 + Foundation-Sec on a triage A/B) | `TRIAGE_MODEL=bugtraceai-apex` |
-| **Exploit code-gen** | `devstral-small-2:24b` (primary) / `qwen2.5-coder:14b` (fast fallback) | Both write valid, runnable PoCs. A/B-validated: Devstral wins on correctness (68 % SWE-bench; emits the canonical sqlmap-GET structure), qwen2.5-coder is faster/lighter | `BRAIN_SCANNER_MODEL=devstral-small-2:24b` |
-
-`brain_scanner.pick_model()` prefers `devstral-small-2:24b` automatically when
-present, else falls back to `qwen2.5-coder:14b`. `brain.py` makes `phi4:14b` the
-default narrator **and** the default triage model (`MODEL_PRIORITY[0]` and
-`TRIAGE_MODEL_PRIORITY[0]`); `bugtraceai-apex` (which resolves to
-`bugtraceai-apex:latest`) sits second in both lists and is only selected when
-pulled or forced via the env var.
+| **Triage** (submit/drop) | `openmythos-27b:latest` | 2026-07-16 false-positive-discipline bench winner (73-judge panel, 5/5 clean runs, 0 invented confirmations). Triage's decisive axis is **FP discipline** — not fabricating a vuln that isn't there. ⚠ Provisional: the bench measured false-positives only, not sensitivity — see the benchmark record. | `TRIAGE_MODEL=openmythos-27b:latest` |
+| **Narration / analysis** | `openmythos-27b:latest` (pinned); code fallback `qwen3:14b` | Faithful, low-fabrication narration. Currently pinned to the triage winner; not separately narration-benched, so the code fallback stays a faithful generic. | `BRAIN_MODEL=openmythos-27b:latest` |
+| **Exploit code-gen** | `qwen3-coder:30b` (pinned) / `qwen2.5-coder:14b` (fast fallback) | A coder **lands grounded PoCs** where triage/analysis models flail in the write-and-run loop. | `BRAIN_SCANNER_MODEL=qwen3-coder:30b` |
+
+Model selection is layered: **`~/.config/vikramaditya/brain.env` file-WINS** (the
+canonical per-machine pin), and the `MODEL_PRIORITY` / `TRIAGE_MODEL_PRIORITY`
+lists in `brain.py` are the code-level **fallback** (first *installed* model wins).
+A pinned model that is **not installed** no longer silently substitutes — it warns
+loudly (and fails under `BRAIN_REQUIRE_PIN=1`), and the model actually used is
+recorded in `brain.MODEL_SELECTION_LOG`. The triage list is now ranked by FP
+discipline (`openmythos-27b` first, then clean alternates `nemesis` / `devstral` /
+`glm`; the offensive-tune `baron-llm` demoted). Full benchmark + caveats:
+[`docs/benchmarks/2026-07-16-triage-fp-discipline.md`](docs/benchmarks/2026-07-16-triage-fp-discipline.md).
+(Historical defaults `phi4` / `bugtraceai-apex` are retired — speed/JSON-optimized,
+not FP-discipline-validated, and were not installed.)
 
 > ⚠️ A `claude-*` tag in your local Ollama is **not** Claude (Claude weights are
 > not downloadable, so any such tag is a mislabeled local model). Always confirm

brain.py

@@ -1060,22 +1060,32 @@ def list_models(self) -> list[str]:
     "mlx-community/Mistral-7B-Instruct-v0.3-4bit",    # 7B fallback
 ]
 
-# Fast triage model priority — phi4:14b first (v9.1.3 benchmark winner)
-# Used by triage_finding() and next_action() where speed > depth
-# 03 May 2026 bench: phi4:14b T1=4.3s vs baron-llm 17s — 4× faster, 100% valid JSON
+# Triage priority — ranked by FALSE-POSITIVE DISCIPLINE (dismiss scanner FPs; never invent/confirm a
+# vuln that isn't there). This objective REVERSES the old "fastest JSON" ordering: speed is secondary.
+# Ranking source: the 2026-07-16 "bench everything" 73-judge FP-discipline panel (5 runs/finalist) on a
+# fixture with 3 known header-sqlmap FALSE-positives. openmythos-27b won 5/5 clean, 0 invented; the
+# RavenX/CyberStrike offensive-tune class INVENTED confirmations; baron-llm (offensive RLHF) is demoted
+# for the same reason. ⚠ PROVISIONAL: that panel had ZERO known TRUE-positives, so it did NOT measure
+# false-NEGATIVES / sensitivity — a model that drops everything scores perfectly here. Re-validate with
+# a true-positive fixture before treating this order as final. See docs/benchmarks/2026-07-16-triage-fp-
+# discipline.md. Note: ~/.config/vikramaditya/brain.env file-WINS, and a set-but-uninstalled TRIAGE_MODEL
+# no longer silently substitutes (see _pin_unavailable). Machine-specific tags below are skipped if absent.
 TRIAGE_MODEL_PRIORITY = [
-    "phi4:14b",                  # ★ v9.1.3 — fastest triage, consistent JSON, no hidden thinking
-    "bugtraceai-apex:latest",       # Zero-refusal security DPO reasoning model
-    "baron-llm:latest",          # BaronLLM — RLHF on offensive security data
-    "aya-expanse:latest",           # Cohere Aya Expanse 8B Multilingual flagship model
-    "gemma4:e4b",                # Gemma 4 4B — fast triage with tool calling
+    "openmythos-27b:latest",     # ★ 2026-07-16 WINNER — 5/5 clean, 0 invented confirmations, reasoning 4.8/5 (dense Qwen3.6-27B, RLVR on vulnerable/fixed code)
+    "nemesis-27b:latest",        # clean alternate (2/2, n small) — dense Qwen3.6-27B offensive QLoRA that KEPT its FP-discipline
+    "devstral-small-2:latest",   # clean alternate (3/3, n small) — coder with strong FP-discipline (coders judge logic, don't assume-vulnerable)
+    "glm47-flash:latest",        # clean alternate (2/2, n small) — non-Qwen MoE, faster than the dense 27Bs
+    "phi4:14b",                  # faithful-narration generic (fast, consistent JSON) — if installed
+    "qwen3:14b",                 # faithful long-context generic — if installed
+    "gemma4:e4b",                # fast small triage with tool calling — if installed
     "vapt-qwen25:latest",        # custom VAPT-tuned fallback
     "vapt-model:latest",
     "qwen3:8b",
+    "baron-llm:latest",          # DEMOTED (was #3): offensive RLHF tune — the 2026-07-16 bench flagged the offensive-tune class for INVENTING confirmations; keep only as a last-ditch triage judge
     "qwen3-coder-64k:latest",    # last resort — big model for triage if nothing else
-    # xploiter/the-xploiter is intentionally NOT in this list — it is WEIGHT-biased
-    # to assert/fabricate vulns (see MODEL_PRIORITY comment) and the 7-Question Gate
-    # is a faithful-evaluation task. It remains an exploit-IDEATION-only fallback.
+    # DROPPED from triage (uninstalled AND not FP-discipline-selected): bugtraceai-apex, aya-expanse.
+    # xploiter/the-xploiter is intentionally NOT in this list — it is WEIGHT-biased to assert/fabricate
+    # vulns (see MODEL_PRIORITY comment) and the 7-Question Gate is a faithful-evaluation task.
 ]
 
 # Token limits — qwen3-coder-64k supports 64K context
@@ -1168,35 +1178,112 @@ def _get_available_models() -> list[str]:
         return []
 
 
+# ── Model-selection provenance + NO silent substitution (friends review 2026-07-16) ─────────────
+# HAZARD (codex+grok): an explicit pin (BRAIN_MODEL / TRIAGE_MODEL / BRAIN_SCANNER_MODEL) that is
+# NOT installed used to be SILENTLY ignored — the picker fell through to the priority list and
+# ultimately available[0]. For a client-facing vuln-triage gate that means "you pinned OpenMythos
+# but you're actually running baron-llm and were never told." A missing pin must be LOUD, and under
+# BRAIN_REQUIRE_PIN=1 (autonomous/client runs) FATAL — never a silent swap. MODEL_SELECTION_LOG
+# records which model+source was actually used so a report can be audited afterwards.
+MODEL_SELECTION_LOG: dict = {}   # role -> {"model", "source", "requested_pin"}
+
+
+def _require_pin() -> bool:
+    return os.environ.get("BRAIN_REQUIRE_PIN", "").strip().lower() in ("1", "true", "yes", "on")
+
+
+def _record_selection(role: str, model, source: str, requested_pin: str = "") -> None:
+    MODEL_SELECTION_LOG[role] = {"model": model, "source": source, "requested_pin": requested_pin or ""}
+
+
+def _match_installed(name: str, available: list) -> str | None:
+    """Resolve a model name against installed tags, honoring Ollama's implicit ``:latest``.
+    So a pin of ``openmythos-27b`` matches an installed ``openmythos-27b:latest`` (and vice-versa).
+    Returns the actual installed tag, or None if genuinely absent."""
+    if not name:
+        return None
+    if name in available:
+        return name
+    if ":" not in name and f"{name}:latest" in available:   # untagged pin -> :latest
+        return f"{name}:latest"
+    if name.endswith(":latest") and name[: -len(":latest")] in available:  # :latest pin -> bare
+        return name[: -len(":latest")]
+    return None
+
+
+# Models that must NEVER be the triage (false-positive) judge even as a last-resort fallback:
+# weight-biased to assert/fabricate vulns, or dropped from triage on evidence. The triage fallback
+# uses this instead of the unrestricted narrator priority (which still contains these).
+_TRIAGE_FALLBACK_EXCLUDE = frozenset({
+    "xploiter/the-xploiter:latest",   # WEIGHT-biased to assert/fabricate — faithful-eval poison
+    "bugtraceai-apex:latest",         # dropped from triage (offensive DPO, not FP-validated)
+    "aya-expanse:latest",             # dropped from triage (multilingual chat, weak judge)
+})
+
+
+def _pin_unavailable(role: str, envvar: str, requested: str, verifiable: bool = True) -> None:
+    """A pin is set but cannot be honored. Warn loudly; raise under BRAIN_REQUIRE_PIN. Never silently swap.
+    verifiable=True  -> the model inventory was read and the pin is genuinely NOT installed.
+    verifiable=False -> the inventory could not be read (Ollama unreachable / empty) — pin UNVERIFIABLE."""
+    if verifiable:
+        state, fix = "is NOT installed", f"`ollama pull {requested}` or correct the pin"
+    else:
+        state, fix = ("could NOT be verified (Ollama returned no models / is unreachable)",
+                      "start Ollama / check OLLAMA_HOST, or correct the pin")
+    msg = (f"PINNED {role} model {requested!r} ({envvar}) {state} — refusing to SILENTLY substitute a "
+           f"different model. Fix: {fix} in ~/.config/vikramaditya/brain.env.")
+    if _require_pin():
+        raise RuntimeError(f"[brain] {msg} (BRAIN_REQUIRE_PIN=1 -> will not run on a fallback model.)")
+    sys.stderr.write(f"{YELLOW}{BOLD}[!] {msg} Falling back — VERIFY the model actually used "
+                     f"(brain.MODEL_SELECTION_LOG) before trusting client output.{NC}\n")
+    sys.stderr.flush()
+
+
 def _pick_model(preferred: str = None) -> str | None:
     """Return the best available model from priority list.
 
     v9.1.4 — env override: BRAIN_MODEL=<name> forces a specific model
     (used by A/B benchmarks, per-engagement model swap without code edits).
+    v10.7 — a set-but-uninstalled BRAIN_MODEL no longer silently substitutes (see _pin_unavailable).
     """
     available = _get_available_models()
-    if not available:
-        return None
 
-    # v9.1.4 env override takes precedence over caller's preferred arg
+    # v9.1.4 env override takes precedence over caller's preferred arg. Check the PIN FIRST — before the
+    # empty-inventory early-return — so a set-but-unhonorable pin is always loud/strict, even if Ollama
+    # returned nothing (the realistic outage case). :latest aliases resolve via _match_installed.
     env_override = os.environ.get("BRAIN_MODEL", "").strip()
-    if env_override and env_override in available:
-        return env_override
+    pin_missing = ""
+    if env_override:
+        match = _match_installed(env_override, available) if available else None
+        if match:
+            _record_selection("narrator", match, "pinned", env_override)
+            return match
+        _pin_unavailable("narrator", "BRAIN_MODEL", env_override, verifiable=bool(available))
+        pin_missing = env_override
+
+    if not available:
+        _record_selection("narrator", None, "no-models", pin_missing)
+        return None
 
     if preferred:
-        # exact match first
-        if preferred in available:
-            return preferred
+        match = _match_installed(preferred, available)
+        if match:
+            _record_selection("narrator", match, "pin-missing-fallback" if pin_missing else "preferred", pin_missing)
+            return match
         # prefix match (e.g. "qwen3" matches "qwen3:8b")
         matches = [m for m in available if m.startswith(preferred)]
         if matches:
+            _record_selection("narrator", matches[0], "pin-missing-fallback" if pin_missing else "preferred", pin_missing)
             return matches[0]
 
     for candidate in MODEL_PRIORITY:
-        if candidate in available:
-            return candidate
+        match = _match_installed(candidate, available)
+        if match:
+            _record_selection("narrator", match, "pin-missing-priority" if pin_missing else "priority", pin_missing)
+            return match
 
     # Last resort: first available model
+    _record_selection("narrator", available[0], "pin-missing-last-resort" if pin_missing else "last-resort", pin_missing)
     return available[0]
 
 
@@ -1258,22 +1345,45 @@ def _technique_hint(finding_description: str) -> str:
 
 
 def _pick_triage_model(preferred: str = None) -> str | None:
-    """Return the best fast triage model — prefers BaronLLM when installed.
+    """Return the best triage model — ranked by FALSE-POSITIVE DISCIPLINE (see TRIAGE_MODEL_PRIORITY).
 
     v9.1.4 — TRIAGE_MODEL=<name> env var overrides for A/B testing.
+    v10.7 — a set-but-uninstalled TRIAGE_MODEL no longer silently substitutes (see _pin_unavailable).
     """
     available = _get_available_models()
+
+    env_override = os.environ.get("TRIAGE_MODEL", "").strip()
+    pin_missing = ""
+    if env_override:
+        match = _match_installed(env_override, available) if available else None
+        if match:
+            _record_selection("triage", match, "pinned", env_override)
+            return match
+        _pin_unavailable("triage", "TRIAGE_MODEL", env_override, verifiable=bool(available))
+        pin_missing = env_override
+
     if not available:
+        _record_selection("triage", None, "no-models", pin_missing)
         return None
-    env_override = os.environ.get("TRIAGE_MODEL", "").strip()
-    if env_override and env_override in available:
-        return env_override
-    if preferred and preferred in available:
-        return preferred
+
+    if preferred:
+        match = _match_installed(preferred, available)
+        if match:
+            _record_selection("triage", match, "pin-missing-fallback" if pin_missing else "preferred", pin_missing)
+            return match
     for candidate in TRIAGE_MODEL_PRIORITY:
-        if candidate in available:
-            return candidate
-    return _pick_model()  # fall back to analysis model
+        match = _match_installed(candidate, available)
+        if match:
+            _record_selection("triage", match, "pin-missing-priority" if pin_missing else "priority", pin_missing)
+            return match
+    # No configured triage model installed. Fall back to a triage-SAFE available model — NOT the
+    # unrestricted narrator priority (_pick_model), which still contains assert/invent-biased tags
+    # (xploiter/bugtraceai/aya). Disable triage (return None) rather than let a fabrication-biased
+    # model become the false-positive judge. This also avoids clobbering the narrator provenance entry.
+    safe = [m for m in available if m not in _TRIAGE_FALLBACK_EXCLUDE]
+    m = safe[0] if safe else None
+    _record_selection("triage", m, ("pin-missing-" if pin_missing else "") + ("safe-fallback" if m else "none-safe"), pin_missing)
+    return m
 
 
 class Brain: