feat(review): gate surface entries on live grounding and functional-surface checks (#8908, #8909) (#9255)

* feat(review): gate surface entries on live grounding and functional-surface checks (#8908, #8909)

`computeGrounding` and `probeFunctionalSurface` have been fully implemented and
unit-tested in registry-logic.ts since the metagraphed port, but neither had a
single caller outside its own test file. The live surface-entry gate
(assessSurfaceEntry -> runSurfaceReview) merged a submission having never once
looked at what its URLs actually serve: it validated that `url`/`source_url`
were well-formed public HTTPS/WSS URLs and stopped there.

Add the missing half — the fetch plumbing and the gating decision — without
touching either primitive:

- surface-verification.ts: SSRF-guarded, redirect-following, truncation-tolerant
  probes of the entry's `source_url` and `url`; evidence extraction (title +
  tag-stripped text, script/style dropped so bundled code cannot forge a signal);
  and the close-vs-hold policy.
- orchestrator: one optional injected `verifyEntry` hook, applied per appended
  entry, only to entries that already passed static validation. The orchestrator
  stays pure and domain-agnostic — any registry can supply its own verifier.
- content-lane-wire: builds the verifier when the new flag is on.

Policy. A CONFIRMED functional failure (a 2xx whose body demonstrably is not the
declared surface — an `openapi` url serving HTML) CLOSES: an objective,
reproducible, trivially-fixable fact, in the same class as the shape violations
the entry validator already closes on. Unconfirmed grounding HOLDS rather than
closes — it is a heuristic over page text, and a legitimate surface can fail it,
so closing on it would one-shot-close good contributions. It no longer merges,
which is the actual #8908 gap.

Three-state, never fail-open. Each check resolves to pass / fail / inconclusive
and these stay strictly distinct: an unreachable probe, a 2xx with an unreadable
or empty body, a missing source, or a throwing verifier all HOLD with their own
reason code, never a pass. A check that silently passes when it could not run
launders "we didn't look" into "we verified".

Flag-gated and OFF by default, on its own flag rather than the content lane's:
this is the lane's first outbound request to submitter-controlled URLs and it can
hold or close submissions that merge today, so both need independent rollback.
Flag-off, no probe is made and the verdict is byte-identical.

* docs(selfhost): document the surface-verification flag in tuning and privacy-security

* test(engine): cover the surface-verification flag in the engine's own node:test suite

The host vitest test for isSurfaceVerificationEnabled/isContentLaneEnabled
(test/unit/content-lane-flag.test.ts) doesn't count toward the engine
package's own Codecov flag -- that's measured from packages/loopover-engine's
node:test suite via c8 (#9064), which had no equivalent test for this file.

Author: JSONbored

apps/loopover-ui/content/docs/privacy-security.mdx

@@ -68,6 +68,7 @@ LOOPOVER_REVIEW_OPS="true" # read-only anomaly scan + outcome stats endpoint
 LOOPOVER_REVIEW_SELFTUNE="true" # self-tightening tuning loop, never loosens
 LOOPOVER_REVIEW_PARITY_AUDIT="true" # shadow-record gate-decision parity readiness
 LOOPOVER_REVIEW_CONTENT_LANE="true" # dedicated content/registry-repo review lane
+LOOPOVER_REVIEW_SURFACE_VERIFICATION="true" # fetch + verify registry surface entries (needs the content lane too)
 LOOPOVER_REVIEW_DRAFT="true" # public draft-submission (contributor fork PR) flow`}
 />
 

apps/loopover-ui/content/docs/tuning.mdx

@@ -165,6 +165,15 @@ any per-PR feature to run on a given repo. So a per-PR feature activates only wh
   registries) through the dedicated content lane — duplicate detection, source-evidence
   reachability, security scanning, scope classification, registry grounding — instead of the
   code gate. Global.
+- `LOOPOVER_REVIEW_SURFACE_VERIFICATION` — live verification of registry surface
+  entries, on top of the content lane above (both must be on). An entry that passes the
+  static shape/safety checks is additionally fetched: its declared source is checked for
+  evidence corroborating the claimed netuid, owner, or host, and an `openapi` /
+  `subnet-api` / `sse` entry is probed to confirm its URL really serves the interface its
+  `kind` declares. A confirmed not-served surface **closes**; unverified corroboration and
+  any **inconclusive** probe **hold for review** — an inconclusive check is never reported
+  as a pass. This is the only content-lane capability that makes outbound requests to
+  submitter-supplied URLs, so it is a separate flag you can roll back on its own. Global.
 - `LOOPOVER_REVIEW_DRAFT` — the public draft-submission flow (the
   `/v1/drafts` endpoints: contributor draft → GitHub OAuth → fork PR). With the
   flag off every draft endpoint 404s. Requires the

packages/loopover-engine/src/review/content-lane/flag.ts

@@ -15,10 +15,27 @@
 export interface ContentLaneEnv {
   /** When truthy ("1"/"true"/"on"/"yes"), the content lane is enabled. Default OFF. */
   LOOPOVER_REVIEW_CONTENT_LANE?: string;
+  /** When truthy, a surface entry that passes STATIC validation is ADDITIONALLY verified against live content
+   *  (#8908 evidence grounding, #8909 functional-surface probing) before it can merge. Default OFF. */
+  LOOPOVER_REVIEW_SURFACE_VERIFICATION?: string;
 }
 
 /** Is the content lane enabled? Default OFF — only a recognized truthy flag turns it on. */
 export function isContentLaneEnabled(env: ContentLaneEnv | undefined | null): boolean {
   if (!env) return false;
   return /^(1|true|yes|on)$/i.test((env.LOOPOVER_REVIEW_CONTENT_LANE ?? "").trim());
 }
+
+/**
+ * Is LIVE surface-entry verification enabled (#8908, #8909)? Default OFF, and deliberately its OWN flag rather
+ * than riding on LOOPOVER_REVIEW_CONTENT_LANE: it is the first thing in this lane that makes OUTBOUND requests
+ * to submitter-controlled URLs, and it can HOLD or CLOSE submissions that merge today. Both properties need to
+ * be rollable back on their own without taking the whole (already-cutover) content lane down with them.
+ *
+ * The caller ANDs this with the lane's existing activation, so verification runs only where a RegistryLaneSpec
+ * already resolved — i.e. it inherits the per-repo scoping for free and needs no config-surface of its own.
+ */
+export function isSurfaceVerificationEnabled(env: ContentLaneEnv | undefined | null): boolean {
+  if (!env) return false;
+  return /^(1|true|yes|on)$/i.test((env.LOOPOVER_REVIEW_SURFACE_VERIFICATION ?? "").trim());
+}

packages/loopover-engine/test/content-lane-flag.test.ts

@@ -0,0 +1,49 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+
+import { isContentLaneEnabled, isSurfaceVerificationEnabled } from "../dist/review/content-lane/flag.js";
+
+test("isContentLaneEnabled is OFF by default (unset / empty / undefined env)", () => {
+  assert.equal(isContentLaneEnabled(undefined), false);
+  assert.equal(isContentLaneEnabled(null), false);
+  assert.equal(isContentLaneEnabled({}), false);
+  assert.equal(isContentLaneEnabled({ LOOPOVER_REVIEW_CONTENT_LANE: "" }), false);
+});
+
+test("isContentLaneEnabled is ON for recognized truthy values (case/whitespace insensitive)", () => {
+  for (const v of ["1", "true", "on", "yes", "TRUE", " On ", "Yes"]) {
+    assert.equal(isContentLaneEnabled({ LOOPOVER_REVIEW_CONTENT_LANE: v }), true);
+  }
+});
+
+test("isContentLaneEnabled is OFF for non-truthy strings", () => {
+  for (const v of ["0", "false", "off", "no", "enabled", "maybe"]) {
+    assert.equal(isContentLaneEnabled({ LOOPOVER_REVIEW_CONTENT_LANE: v }), false);
+  }
+});
+
+// #8908/#8909: live surface verification is its OWN flag, independent of the lane's, so the first
+// outbound-probe behavior in this lane can be rolled back without taking the whole lane down.
+test("isSurfaceVerificationEnabled is OFF by default (unset / empty / undefined env)", () => {
+  assert.equal(isSurfaceVerificationEnabled(undefined), false);
+  assert.equal(isSurfaceVerificationEnabled(null), false);
+  assert.equal(isSurfaceVerificationEnabled({}), false);
+  assert.equal(isSurfaceVerificationEnabled({ LOOPOVER_REVIEW_SURFACE_VERIFICATION: "" }), false);
+});
+
+test("isSurfaceVerificationEnabled is ON for recognized truthy values (case/whitespace insensitive)", () => {
+  for (const v of ["1", "true", "on", "yes", "TRUE", " On ", "Yes"]) {
+    assert.equal(isSurfaceVerificationEnabled({ LOOPOVER_REVIEW_SURFACE_VERIFICATION: v }), true);
+  }
+});
+
+test("isSurfaceVerificationEnabled is OFF for non-truthy strings", () => {
+  for (const v of ["0", "false", "off", "no", "enabled", "maybe"]) {
+    assert.equal(isSurfaceVerificationEnabled({ LOOPOVER_REVIEW_SURFACE_VERIFICATION: v }), false);
+  }
+});
+
+test("isSurfaceVerificationEnabled is independent of the content-lane flag in BOTH directions", () => {
+  assert.equal(isSurfaceVerificationEnabled({ LOOPOVER_REVIEW_CONTENT_LANE: "true" }), false);
+  assert.equal(isContentLaneEnabled({ LOOPOVER_REVIEW_SURFACE_VERIFICATION: "true" }), false);
+});

src/env.d.ts

@@ -516,6 +516,16 @@ declare global {
      *  gate disposition byte-identical. AI-FREE (pure structured-data adjudication), so independent of the AI
      *  reviewer; a generic hard blocker (e.g. a committed secret) is always preserved over a surface "merge". */
     LOOPOVER_REVIEW_CONTENT_LANE?: string;
+    /** Convergence (surface LIVE VERIFICATION, #8908/#8909): when truthy *AND* the surface lane above is active
+     *  for the repo, a surface entry that passes STATIC validation is additionally checked against what its URLs
+     *  actually serve — `computeGrounding` over the fetched source/target evidence (is the declared netuid /
+     *  owner / host corroborated at all?) and `probeFunctionalSurface` for the openapi/subnet-api/sse kinds (does
+     *  the url serve the interface its `kind` claims?). A CONFIRMED not-served functional surface CLOSES;
+     *  unconfirmed grounding and any INCONCLUSIVE probe HOLD for review — an inconclusive check is never reported
+     *  as a pass. Default OFF: unset/false makes no outbound probe and leaves the surface verdict byte-identical.
+     *  Its own flag, separate from the lane's, so the first outbound-fetch behavior in this lane can be rolled
+     *  back on its own — see review/content-lane/surface-verification. */
+    LOOPOVER_REVIEW_SURFACE_VERIFICATION?: string;
     /** Convergence (self-improve / auto-tune): when truthy, the ported self-improvement loop
      *  (src/review/auto-tune.ts + auto-apply.ts) runs on the cron tick over loopover's OWN review-outcome
      *  data — it computes tuning recommendations, SHADOW-SOAKS any STRICTLY-TIGHTENING recommendation in the

src/review/content-lane-wire.ts

@@ -37,10 +37,11 @@
 //     genuinely critical finding, or one of these other configured gates, still wins outright).
 import { AI_JUDGMENT_BLOCKER_CODES, type GateCheckEvaluation, isAiJudgmentOnlyFailure, isDuplicateOnlyFailure } from "../rules/advisory";
 import { LOOPOVER_GATE_CHECK_NAME } from "./check-names";
-import { isContentLaneEnabled } from "./content-lane/flag";
+import { isContentLaneEnabled, isSurfaceVerificationEnabled } from "./content-lane/flag";
 import { runSurfaceReview, type SurfaceReviewInput, type SurfaceReviewResult } from "./content-lane/orchestrator";
 import type { RegistryLaneSpec } from "./content-lane/registry-logic";
 import { registeredValidatorIds, resolveRegistryLaneSpec, unregisteredValidatorId } from "./content-lane/spec-resolver";
+import { makeSurfaceEntryVerifier } from "./content-lane/surface-verification";
 import { makeGithubFileFetcher } from "./grounding-wire";
 import { MAX_FETCH_CHARS } from "./review-grounding";
 import type { FocusManifest } from "../signals/focus-manifest";
@@ -185,6 +186,7 @@ export async function runRegistrySurfaceGate(
     files: { path: string; status?: string | null | undefined }[];
   },
   loadFileOverride?: SurfaceReviewInput["loadFile"],
+  verifyEntryOverride?: SurfaceReviewInput["verifyEntry"],
 ): Promise<GateCheckEvaluation | null> {
   let fetcherPromise: ReturnType<typeof makeGithubFileFetcher> | null = null;
   const githubLoad = async (path: string, ref: "head" | "base"): Promise<string | null> => {
@@ -219,10 +221,15 @@ export async function runRegistrySurfaceGate(
     if (ref === "base" && content === null && statusByPath.get(path) === "modified") deferUnreadable = true;
     return content;
   };
+  // #8908/#8909: live verification is its OWN flag on top of the lane's activation, so it can be rolled back
+  // without taking the (already-cutover) surface lane down. Flag-OFF, `verifyEntry` is undefined, the
+  // orchestrator runs no verification and makes no outbound probe, and the verdict is byte-identical to today.
+  const verifyEntry = verifyEntryOverride ?? (isSurfaceVerificationEnabled(env) ? makeSurfaceEntryVerifier() : undefined);
   const result = await runSurfaceReview(spec, {
     changedFiles: args.files.map((file) => file.path),
     loadFile,
     opts: { secretsScan: true, sourceUrlValidation: true },
+    ...(verifyEntry ? { verifyEntry } : {}),
   });
   if (result === null) return null; // not a registry submission → the generic gate applies
   if (deferUnreadable) return null; // a fetch blip on a file that must be readable → defer, never auto-close

src/review/content-lane/index.ts

@@ -116,4 +116,20 @@ export {
   type ProviderLike,
   type Verdict,
 } from "./registry-logic";
+export {
+  fetchSurfaceProbe,
+  makeSurfaceEntryVerifier,
+  probeToEvidence,
+  verifySurfaceEntry,
+  FUNCTIONAL_INCONCLUSIVE_REASON,
+  FUNCTIONAL_NOT_SERVED_REASON,
+  GROUNDING_INCONCLUSIVE_REASON,
+  GROUNDING_UNCONFIRMED_REASON,
+  MAX_PROBE_BODY_CHARS,
+  SURFACE_GROUNDING_MIN_STRONG,
+  type SurfaceCheckOutcome,
+  type SurfaceCheckResult,
+  type SurfaceEntryVerification,
+  type SurfaceProbe,
+} from "./surface-verification";
 export { runSurfaceReview, diffAppendedSurfaceEntries, type SurfaceReviewInput, type SurfaceReviewResult } from "./orchestrator";

src/review/content-lane/orchestrator.ts

@@ -30,6 +30,13 @@ export interface SurfaceReviewInput {
   /** Loads decoded file content at a ref; injected so unit tests need no network. Returns null when absent. */
   loadFile: (path: string, ref: "head" | "base") => Promise<string | null>;
   opts?: { secretsScan?: boolean; sourceUrlValidation?: boolean };
+  /** Optional LIVE content verification for an entry that already passed static validation (#8908, #8909) —
+   *  injected I/O, exactly like `loadFile`, so the orchestrator stays pure and domain-agnostic and any registry
+   *  can supply its own verifier (metagraphed's is `makeSurfaceEntryVerifier` in ./surface-verification).
+   *  Returns an OVERRIDING Assessment when verification downgrades the entry (hold or close), or `null` when it
+   *  confirms it (the static assessment stands). Omitted ⇒ no verification runs and no fetch is made, so the
+   *  verdict is byte-identical to the pre-#8908 lane. */
+  verifyEntry?: (entry: unknown) => Promise<Assessment | null>;
 }
 
 export interface SurfaceReviewResult {
@@ -192,6 +199,42 @@ function pickAggregateAssessment(assessments: Assessment[]): Assessment {
   return first as Assessment;
 }
 
+/**
+ * Apply the injected live-content verifier (#8908, #8909) to every appended entry that passed STATIC validation,
+ * and return the per-entry assessments with any downgrade folded in.
+ *
+ * Only "merged" entries are verified, for two reasons: an entry already closing or held has its verdict decided
+ * (verification could only ever confirm it), and skipping them means an invalid submission — the common bad case
+ * — pays for no network I/O at all. The surviving entries are verified CONCURRENTLY (independent URLs, no data
+ * dependency), mirroring how this function's caller already parallelizes its GitHub reads.
+ *
+ * A verifier that THROWS must never take the review down or, worse, be swallowed into a pass: the entry is held
+ * for review instead. `makeSurfaceEntryVerifier` is itself written not to throw, so this is a belt-and-braces
+ * guard against a future/third-party verifier that is less careful.
+ */
+async function verifyMergedEntries(
+  staticAssessments: Assessment[],
+  appendedEntries: readonly unknown[],
+  verifyEntry: SurfaceReviewInput["verifyEntry"],
+): Promise<Assessment[]> {
+  if (!verifyEntry) return staticAssessments;
+  return await Promise.all(
+    staticAssessments.map(async (assessment, idx) => {
+      if (assessment.verdict !== "merged") return assessment;
+      try {
+        return (await verifyEntry(appendedEntries[idx])) ?? assessment;
+      } catch {
+        return {
+          verdict: "manual-review" as const,
+          summary: "Live verification of this surface entry could not be completed — routing to review rather than accepting an unverified entry.",
+          candidate: assessment.candidate,
+          reason: "verification-error",
+        };
+      }
+    }),
+  );
+}
+
 /**
  * Adjudication policy (deterministic, DECISIVE): the overwhelming majority of outcomes are merge or close —
  * manual review is the rare exception. A clean valid submission MERGES; anything invalid or non-standard
@@ -279,9 +322,8 @@ export async function runSurfaceReview(spec: RegistryLaneSpec, input: SurfaceRev
     return { verdict: "manual", summary: NO_VALIDATOR_ENTRY_SUMMARY };
   }
   const headDoc = safeParseJson(headRaw);
-  const assessment = pickAggregateAssessment(
-    appendedEntries.map((appendedEntry) => assessEntry(headDoc, { ...input.opts, appendedEntry })),
-  );
+  const staticAssessments = appendedEntries.map((appendedEntry) => assessEntry(headDoc, { ...input.opts, appendedEntry }));
+  const assessment = pickAggregateAssessment(await verifyMergedEntries(staticAssessments, appendedEntries, input.verifyEntry));
   if (companionProviderFile !== null) {
     // Guaranteed non-null: the no-validator short-circuit above already returned when this spec lacks one.
     const assessProvider = spec.assessProviderEntry as NonNullable<RegistryLaneSpec["assessProviderEntry"]>;