Summary
applyDeterministicResponsePolicy in convex/domains/redesign/chatRuns.ts sanitizes "best/strongest source|claim" superlatives only when zero URL-backed evidence rows exist run-wide (hasSupportedUrl === false). In a mixed run — one valid URL row + several cached section labels (e.g. Setup, Rising Action) — hasSupportedUrl is true, so the superlative passes through unsanitized even when the rendered "best source" is a label with no URL.
Impact
On a paid runtime path, the answer can assert "strongest supported claim with its best source" while the cited source is a cached label. Honesty gap surfaced by the 2026-07-16 production audit.
Status
Being fixed in a follow-up to PR #565 via a sentence-level citation gate: the sentence making the superlative must carry a [N] marker resolving to a URL-backed evidence row, else it is rewritten and the source-needed limitation is appended. This issue tracks that work.
Files
convex/domains/redesign/chatRuns.ts — applyDeterministicResponsePolicy (~L943), sanitizeUnsupportedSuperlatives (~L896)
Summary
applyDeterministicResponsePolicyinconvex/domains/redesign/chatRuns.tssanitizes "best/strongest source|claim" superlatives only when zero URL-backed evidence rows exist run-wide (hasSupportedUrl === false). In a mixed run — one valid URL row + several cached section labels (e.g.Setup,Rising Action) —hasSupportedUrlis true, so the superlative passes through unsanitized even when the rendered "best source" is a label with no URL.Impact
On a paid runtime path, the answer can assert "strongest supported claim with its best source" while the cited source is a cached label. Honesty gap surfaced by the 2026-07-16 production audit.
Status
Being fixed in a follow-up to PR #565 via a sentence-level citation gate: the sentence making the superlative must carry a
[N]marker resolving to a URL-backed evidence row, else it is rewritten and the source-needed limitation is appended. This issue tracks that work.Files
convex/domains/redesign/chatRuns.ts—applyDeterministicResponsePolicy(~L943),sanitizeUnsupportedSuperlatives(~L896)