Make mention analysis replayable: persist extracted text, recompute on brand changes

[jrhizor]

Fixes #329

What

Makes mention analysis replayable instead of frozen at write time, and fixes the substring-matching false positives.

Schema (packages/lib/src/db/schema.ts, migration 0010)

• prompt_runs.text_content (text, nullable) — the provider-extracted answer text, now persisted instead of being re-parsed from raw_output on every read.
• prompt_runs.analyzed_at (timestamptz, nullable) — when mentions were last computed (write time or last re-analysis).

Note: the migration was generated with drizzle-kit generate (offline) but not run — it needs npx drizzle-kit migrate from packages/lib at deploy time.

Consolidated + fixed matcher

• New @workspace/lib/mention-analysis module owns analyzeMentions() / extractDomainFromUrl() — the worker's private copy is deleted, so worker, jobs, and future backfills share one implementation.
• Names (brand, aliases, competitors) match on word boundaries ((^|\W)name(\W|$), case-insensitive): a brand named "Box" no longer matches "toolbox" or "Dropbox". Domains keep substring semantics (they're unambiguous tokens). \W instead of \b so names like "C++" still match.
• isPromptBranded() in tag-utils.ts (branded/unbranded system tags) gets the same fix: word-boundary for the name and domain-without-TLD, substring for the full domain.

Write path

• process-prompt persists text_content alongside raw_output and stamps analyzed_at.
• If extraction yields empty/missing text (or a "No text content found…" sentinel), it logs, reports to Sentry, and stores NULL — never a silently-analyzed empty string (better text extraction error handling #106). New tryExtractTextContent() returns null instead of the human-readable failure sentinels.

Backfill job (backfill-text-content)

• Populates text_content for historical rows from raw_output. Batched (keyset pagination), resumable (WHERE text_content IS NULL), idempotent. Enqueued on worker startup with a singleton key, no-ops once complete.

Re-analysis job (reanalyze-brand)

• Recomputes brand_mentioned, competitors_mentioned, and prompt system_tags for all of a brand's runs from text_content (falling back to extraction, opportunistically backfilling), in batches.
• Enqueued (singleton key per brand, 60s debounce) whenever brand name/aliases/website/domains or competitors change: updateBrandFn, updateCompetitors, addDomainToBrandFn, addDomainToCompetitorFn, createCompetitorFromDomainFn.

Read path

• The prompt run detail view prefers text_content and only falls back to extractTextContent(raw_output) for rows the backfill hasn't reached. The fallback can be removed entirely once the backfill has completed everywhere.

Acceptance criteria from the issue

• ✅ Changing an alias/competitor and saving settings enqueues reanalyze-brand, which updates historical mention data.
• ✅ A brand named "Box" does not match "toolbox" (unit-tested in mention-analysis.test.ts and tag-utils.test.ts).
• ✅ Run display no longer parses raw_output for backfilled rows (fallback kept until backfill completes, per spec §6).

Testing

• pnpm turbo test — 332 tests pass (incl. new mention-analysis.test.ts and word-boundary cases in tag-utils.test.ts).
• pnpm turbo check-types — clean.
• Web lint failures are pre-existing (identical count on a clean tree); no new issues in changed files.
• No DB migrations were run and no DB contents were altered.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
elmo	Ready	Preview, Comment	Jun 12, 2026 2:22am