Skip to content

feat(commands): intent-classification router for unrecognized @gittensory mentions - #5036

Merged
JSONbored merged 2 commits into
mainfrom
feat/intent-classification-router-4596
Jul 11, 2026
Merged

feat(commands): intent-classification router for unrecognized @gittensory mentions#5036
JSONbored merged 2 commits into
mainfrom
feat/intent-classification-router-4596

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Closes #4596

Adds a closed-set intent-classification router: an @gittensory mention whose verb doesn't resolve to a known command, but has real trailing text (e.g. "why is this stuck?"), gets ONE chance to be classified against the existing Q&A command catalog before falling back to the plain did-you-mean hint.

  • Never a generator, always a closed set. The classifier maps free text to the single closest match among 9 existing Q&A commands (preflight, blockers, duplicate-check, next-action, reviewability, repo-fit, packet, ask, chat) or reports no match — it never synthesizes new content, so there's no hallucination surface, only mis-routing.
  • Hard runtime allowlist, not a prompt instruction. isIntentRoutableCommand is a plain Set membership check the classifier's raw output must pass before ever being trusted — a prompt-injection attempt naming an action command (review, gate-override, etc.) is filtered to "no match" regardless of what the model says. Exhaustively tested against every action command, every non-routable Q&A command, and arbitrary/non-string values.
  • On a match, re-dispatches through the exact same path that command already uses — authorization, rate limiting, and rendering all apply unchanged, exactly as if the contributor had typed the exact verb — with a visible "interpreted as @gittensory X" note so a wrong match is immediately correctable.
  • New advisoryAiRouting.intentRouting capability flag (default false), Ollama-only same as chatQa, global + per-repo override.
  • Its own dedicated rate-limit counter (reusing the existing AI-cost-bearing ceiling) so a classification attempt — matched or not — is never left uncounted just because it doesn't resolve to a fixed command name up front.

Part of #4594 (the epic's last remaining sub-issue).

Test plan

  • Exhaustive allowlist test: all 9 routable names, every action command, every non-routable Q&A command, and non-string/malformed values
  • Classifier unit tests: disabled/unavailable/no_match/matched/quota_exceeded/error, plus adversarial prompt-injection attempts naming action commands
  • Parser tests for the new unrecognizedText capture (verb+trailing reconstruction, no-verb-token case, bare mention)
  • Render test for the "interpreted as" note (present/absent) plus the markdown-injection-neutralization regression
  • Full end-to-end webhook dispatch tests: matched re-route, no-match fallback, and disabled-by-default — proving processors.ts actually reaches the classifier and the rewritten command flows through the real pipeline
  • Capability-flag normalization + focus-manifest merge tests (both app and engine-package copies)
  • npm run typecheck, npm run test:coverage (unsharded, 100% on every changed line/branch), npm run test:ci, npm audit --audit-level=moderate all green
  • npm run ui:openapi regenerated and committed
  • npx tsx scripts/check-engine-parity.ts passes

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 11, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
gittensory-ui 6729034 Commit Preview URL

Branch Preview URL
Jul 11 2026, 10:20 AM

…sory mentions

Adds a closed-set intent classifier that maps a free-text natural-language
question to the single closest existing Q&A command (never an action
command), so a contributor who doesn't know the exact verb still gets routed
to the right existing, already-tested answer instead of just a did-you-mean
hint. The classifier's raw output is filtered through a hard runtime
allowlist before ever being trusted, never a prompt instruction alone.

Ollama-only via a new advisoryAiRouting.intentRouting flag (default false,
global + per-repo override, same shape as chatQa). Rate-limited through its
own dedicated counter, reusing the existing AI-cost-bearing ceiling.

Closes #4596
maybeThrottleIntentRouting's body past the initial policy-off check was
never exercised by any test -- the 3 existing #4596 integration tests all
ran under the default (off) policy. Add hold-policy under/over-ceiling and
same-deliveryId redelivery cases, fix a stale AdvisoryAiRoutingConfig test
literal missing chatQaFrontierFallback, and regenerate openapi.json.
@codecov

codecov Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.80952% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 94.37%. Comparing base (5b2334e) to head (6729034).
⚠️ Report is 9 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/queue/processors.ts 95.23% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5036   +/-   ##
=======================================
  Coverage   94.36%   94.37%           
=======================================
  Files         459      460    +1     
  Lines       39285    39366   +81     
  Branches    14329    14360   +31     
=======================================
+ Hits        37072    37152   +80     
  Misses       1558     1558           
- Partials      655      656    +1     
Flag Coverage Δ
shard-1 46.65% <11.90%> (-0.08%) ⬇️
shard-2 33.82% <66.66%> (+0.45%) ⬆️
shard-3 32.24% <5.95%> (+0.75%) ⬆️
shard-4 31.66% <8.33%> (-1.18%) ⬇️
shard-5 33.54% <71.42%> (+0.08%) ⬆️
shard-6 45.03% <10.71%> (-0.08%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
packages/gittensory-engine/src/focus-manifest.ts 99.17% <100.00%> (+<0.01%) ⬆️
...ry-engine/src/review/advisory-ai-routing-config.ts 100.00% <ø> (ø)
src/github/commands.ts 98.28% <100.00%> (+0.01%) ⬆️
src/openapi/schemas.ts 100.00% <ø> (ø)
src/review/advisory-ai-routing-config.ts 100.00% <ø> (ø)
src/services/ai-intent-router.ts 100.00% <100.00%> (ø)
src/signals/focus-manifest.ts 99.65% <100.00%> (+<0.01%) ⬆️
src/types.ts 100.00% <ø> (ø)
src/queue/processors.ts 95.37% <95.23%> (-0.01%) ⬇️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 11, 2026
@loopover-orb

loopover-orb Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Caution

🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

🛑 Gittensory review result - fixes required

Review updated: 2026-07-11 10:29:47 UTC

21 files · 1 AI reviewer · no blockers · readiness 100/100 · CI failing · unstable

🛑 Suggested Action - Manual Review

  • Touches a guarded path — held for manual review: This PR changes guardrail-protected path(s): src/queue/processors.ts (matched src/queue/**), src/services/ai-intent-router.ts (matched src/services/**).

Review summary
AI review could not be completed for this PR head. Gittensory is holding this PR for manual review instead of relying on deterministic signals alone.

Nits — 2 non-blocking
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.

CI checks failing

  • codecov/patch — 98.80% of diff hit (target 99.00%)
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #4596
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 47 registered-repo PR(s), 39 merged, 428 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 47 PR(s), 428 issue(s).
Gate result ⚠️ Not blocking Advisory; not blocking this PR.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, Ruby, Go, JavaScript, MDX, Shell, Solidity
  • Official Gittensor activity: 47 PR(s), 428 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
Visual preview
Route Viewport Before (production) After (this PR's preview) Diff
/ desktop before / after /
/ mobile before / (mobile) after / (mobile)

Click any thumbnail to open the full-size screenshot. Before = production · After = this PR's preview deploy.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 11, 2026
@JSONbored
JSONbored merged commit f8f281b into main Jul 11, 2026
19 of 20 checks passed
@JSONbored
JSONbored deleted the feat/intent-classification-router-4596 branch July 11, 2026 10:46
JSONbored added a commit that referenced this pull request Jul 11, 2026
…ranch (#5056)

#5036's codecov/patch flagged one partial branch in processors.ts:
matchedCommand === "ask" || matchedCommand === "chat" ? command.unrecognizedText
: undefined. The 3 existing #4596 integration tests only ever reroute to
"blockers", so the true side (rerouting to ask/chat specifically) was never
exercised -- meaning the one behavior that actually matters here (does the
contributor's original free text survive the reroute into the command's own
question field, instead of silently dropping it) was untested.
JSONbored added a commit that referenced this pull request Jul 11, 2026
…5060)

* test(commands): cover the intent-router ask/chat question-threading branch

#5036's codecov/patch flagged one partial branch in processors.ts:
matchedCommand === "ask" || matchedCommand === "chat" ? command.unrecognizedText
: undefined. The 3 existing #4596 integration tests only ever reroute to
"blockers", so the true side (rerouting to ask/chat specifically) was never
exercised -- meaning the one behavior that actually matters here (does the
contributor's original free text survive the reroute into the command's own
question field, instead of silently dropping it) was untested.

* fix(selfhost): widen GitHub-native id columns to bigint on Postgres

Every migrations/*.sql column storing a raw GitHub-native numeric id
(installation, account/user, check-run, comment) is declared bare
INTEGER -- fine on SQLite/D1 (a type-affinity hint that already stores
any 64-bit value), but a real 4-byte column on the self-host Postgres
backend. GitHub's comment ids are already past 2^31, confirmed live on
edge-nl-01 via a "value out of range for type integer" insert failure
on github_agent_command_answers.

widenGithubIdColumnsToBigint mirrors the existing
tuneGithubRateLimitObservationsAutovacuum pattern: a Postgres-only,
idempotent ALTER batch run unconditionally after migrations on every
boot, never touching the original (SQLite/D1-correct) migration files.

Closes #5059
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

feat(commands): intent-classification router for unrecognized @gittensory mentions

1 participant